I don't remove it. I have mystery snails that clean it off in one tank, otos in another and in the third tank, the african featherfin and the fish nibble on it.
Personally, I think if your algae is that bad, you might have a problem in your tank, be it fertilizing too much, too much light, etc. Even before I had "algae eaters", I didn't have to rub it down more than once every other month or so.
75: blood parrots, featherfin cat, emperor tetras, turquoise rainbowfish, BN plecos, japanese trapdoor; 46: WCMM, gold inca snails, ghost, bamboo & amano shrimp, kuhli loaches, rummynose & ember tetras, endlers, platies, flame gourami, guppies; 16: pygmy, peppered, loxozonus corys, otos, assassins, RCS ... ~ Research PRIOR to purchase.... ~