My DW did the same thing. It got really thick so I had to scrub it off. I can't prove anything but I feel like when it built up it clouded up my water plus if it's disturbed it sends the particles all over the tank.
I would just scrub it off for good measure but like Mike said no it can't hurt anything besides putting a damper on appearance.
Caleb
Might have a slight obsession with my fish