I know. It's being responsible for a life that you bring home and feeding it and nurturing, and all that mom or dad like stuff. I hear ya.
Replacing fish as they grow ain't that easy. With rock work and other fish to disturb, and the stress of new additions, can be asking for trouble. You'll enjoy it more with established fish, patterns, maintenance and all. Trading fish in every time they grow could easily give you bad experiences too.
I'd really consider taking the fish back, cycling with shrimp, and starting to look at gobies, blennies, cardinalfish, chromis, shrimp, crabs, and snails. They'll provide lots of fun I think.