i would use gloves whenever possible. not so much so you don't get stung, but so you don't harm the coral. we pick up lots of things on our hands in a day, and even soap isn't the best thing to be touching to the surface of a coral.
i had a large sps
colony that i grew from a tiny frag that i touched one day, just to snip a tiny piece off (which i've done many times before), and the next day it was completely dead.
i believe this was from something that was present on my hands. i never removed it from the water.