You would scrub it off. That said, it is still a bad idea. Who knows what kind of toxins can be stuck in those rocks. Things coming through tidal zones carry all kinds of pollutants, just imagine a protein skimmer collection cup as it is what our tidal zones are...but trying to skim out all kinds of chemicals and who knows what else. This is why it is suggested that you go into deep water to even take water from the ocean itself for use in the home aquaria.
This going into that tank you were cycling back in March I'd assume?