5 goldies in 10 gallons! ouch! Good idea on getting that bigger tank!
As others have said, you should only be doing a 50% water change at the absolute maximum, and the fish should never have to leave the tank unless it's exceptional circumstances.
A water change doesn't stress them too much, in fact much of the time they'll find it interesting!
One thing, is your water, decorations, gravel etc becoming covered in grime all the time? It could be that you're lighting your tank too much, which is encouraging algae growth.
The fish don't need the lights on, they prefer the dark, only have your fish tank lights on for periods when you want to look at the fish. Get a timer to make sure that when you are in bed, at work etc, that the tank lights are off.