I used to have quite a bit actually when I just let the babies roam free right after birth. I would get a lot of dead ones sinking to the bottom and then I would have to reach in there and fish them out.
The only time I really had a problem with deaths was when I didn't have time to turn the lights on. I read somewhere that if you don't give them enough light they won't grow properly i.e. crooked spines.
It's definitely easier to have these because when they're up closer to the top you can easily suck out uneaten food, debris, poop, etc.
Sometimes I use them to grow plants that have been damaged or seem to be dying because they're closer to the light in the box.