As elecrobes says, the sludge is almost certainly a) fish waste or b) algae. Algae is *not* uncommon, and can appear in any tank! It indicates that you have too many nutrients in your tank. Lighting is an issue, but algae will grow under even weak light if it has the necessary nutrients (i.e. nitrogenous compounds such as ammonia, nitrites, nitrates).
Do yourself a favor and buy a gravel vacuum for your tank. They cost about $7-14 depending on the size (look carefully on the package to see what size is appropriate for your tank). The Gravel vac will suck up fish crap and uneaten food from on top of and in between the pieces of gravel. It's an ingenious design, as the widened body of the vacuum sucks up small particles of waste, and will not suck up your gravel! Don't be intimidated by the name--it's a simple syphon tube with a little bit extra.
Also, how often do you do water changes? You should be changing 15-25% of your water once a week.