It looks like the green slime is green cyanobacteria to me. (Most folks see the red / burgundy version.) Carey's recommendation is spot on. You need manual removal (vacuum it out with water changes) and work to control phosphates and nitrates. In some cases, the test kits don't pick up the nitrates and phosphates because the stuff absorbs it as fast as it is produced. Make sure you are using RODI
water with zero nitrates or phosphates coming from the source.
I think your gorgonian can recover as long as the algae stays off of it. I'm pretty sure yours is not photosynthetic but is a filter feeder.
I had a really bad case of the red version and ended up removing my live rock one piece at a time and scrubbing it in a bucket with a brush along with massive water changes to clear it up. The source of my issue was high nitrates and some phosphates coming from tapwater, which is a big NONO.