If its hard to scrape, it's probably another form of coralline. I have some spots of a dark red algea that seems kind of fuzzy too and will not come off the rocks. I've just accepted that some forms aren't going to be as nice looking as the pinks, purples, oranges, etc.
I've heard of some people refer to them as stains and try to remove them, but I don't see the point. IMO, it's just another thing that comes to life in our systems,