I have it flush with the floor horizontaly. I dont know if having it vertical makes a difference. I dont see how significant that can be but give it a shot. Try disconnecting it for a bit and reconnect it.
If it doesn't work, use a Hammer.