It shouldnt come back as fast as you say, whats your cleanng schedule like? You could add a little power head or increase filtration for more waater movement. As far as i know its just fish,food waste etc.. it really wont harm anything to much except make nitrates go up some and be an eye sore.
Okay I do a 50% water change a week. I struggle to get to it with my siphon because of plants but I can still get some. I have two filter an ac 30 and a c3,