My only issue with hermits is that they're not the most dainty things when dragging themselves across your landscape. I lost two corals early on due to damage that I'm 99% sure was inflicted by one of my larger hermits dragging/banging its shell around. Since then, I've gotten smarter at coral placement AND I've gotten rid of all but about 2 or the most docile ones that seem to understand where the "no go" zones are! I've never seen them purposefully damage or nip at corals - hermits, that is. However, I just saw a thread on another forum (with pictures) of an Emerald Crab ripping the heck out of an Acan. It wasn't going after food, but it sounds like it was just ripping it up just because it could!
I think hermits are a good thing to have in tanks, but just not in the numbers normally recommended in the typical clean up crew "packages" that are advertised. Yeah... they'll kill a snail now and then, but hey... that's nature at work.