Kuhlis are scavengers. They search the whole tank in search of food. Yes, they might stay away for a bit after they got a sniff of the sponge. But its their instinct to go around looking for food and will just end up back there. If you take that sponge stuff from the plants and rub it between your fingers you'll notice it has some irritating feeling to it. Now imagine what it must be like for a 2 inch fish.
What I do is rinse it off under the tap. It does get most of it off. I agree its near impossible to get all of it off. But we do our best. afterwards you can make sure the gravel/sand covers it. That should keep the problem to a minimum.