If you have a clean up crew of snails/crabs they'll pick most of the junk out of the tank for you making the canister filter really nothing more then a bacteria colony. You could remove it, but I would do it slowly. If there is bacteria growing in there and you yank it out, you could throw the tanks bio filter out of whack and it would have to grow new bacteria.
If it has 2 filter pads, try removing one and check your parameters after a couple days. If it's reading 0 amm. and 0 nitrite, then pull the second one. Then check the parameters again in a few days, then if it's still reading fine, go ahead and remove the filter all together. If during those steps you see the parameters shift, wait a week until you yank out the next filter pad to give your colony a chance to regrow somewhere else.
I wouldn't throw it away though, its good to have in storage in case you need to run carbon or some other media in the future.
edit: I also see your a fellow upstate NYer, score another for us!