The first thing that would signal that your BTA is dying would be a shrivled, lifeless apperance. Remember though, BTA's and other anemone types will shrivel up when releasing waste, at which point you will see liquid waste or slime rising from thier mouth/anus.
What would really make it obvious that your anemone is dying would be smell. Anemones actually decompose as they did, releasing a horrible smell into the water, one that can fill an entire room with utter stench. Other than that you would witness the actual decomposition of the bast and body, and likely its mouth would remain unguarded by tentacles and open.
If your BTA is dying, and dies it will release tons of toxins into the water, raising nitrites, nitrates and ammonia, and potentially wiping out your entire tank.