What kind of worms? Bristle worms don't kill crabs or mine would have been dead a long time ago, I have several about 6 inches long and they never bother anything that is alive. The could be killing them is the SG
. One other thing that kills them is each other I have split up more than one crab fight when they were fighting over a shell. Molting is another reason they die when they molt another hermit will get it or they don't complete the molt.