I think what you are referring to are spirorbid polychaetes. Basically small feather dusters, however, they won't get any bigger than what you are seeing now. They aren't really "babies", but a type of small tube worm/duster.
Here's a pic:
They are harmless filter feeders. You tend to see a ton of them, then they sporadically dissapear. More common in newer tanks (though I'm not sure why).
There's some info. on them in this article:
Reefkeeper's Guide to Invertebrate Zoology: Part 10: Sedentary Polychaetes, Con't.
See the section on Spirorbidae