I've got a few patches of colonial hydroids, and while they're not desirable, they're not the end of the earth either. BUT... if that's what I'm seeing on that rock, if I was you, I'd just throw the rock out if that's possible. If they're colonial hydroids, you've got a TON of them. Of if you like the rock and want to keep it, then boil it and let it dry out.
Only thing that makes me think it's not colonial hydroids is the comment you made about the "vine-like" structure. That's not how colonial hydroids grow/spread. A closer and clearer picture would really help positively ID the stuff. A vine like structure sounds more like zoanthid/palythoas and they could just be some of those brown/yellow polyps.