I honestly couldnt tell you how long it might take to break down, it could be a day or two.
Goldfish can go a decent amount of time without food but what concerns me is the fish needs to be able to pass water through its gills to breath.
I know you are concerned with stress but the stress you might cause for a minute or two pushing the pea out through the gills with something small and blunt might be better than the fish stressing for a few days with it in its mouth.
You could also try very gently squeezing the fish where the pea is to mush it a little to be able to grab it with tweezers or allow him to be able to spit it out sooner.
These are all suggestions and I cant vouche for any of them being the best thing to do.
Also, do you de-shell you peas and boil them? I don't feed peas to my goldie cause it dirties the water and is a pain but I know you need to make sure they are very soft when you feed them. Have you thought about moving away from peas to something healthier like Mysis Shrimp? Its very good for the goldie, cant choke on it (if you dethaw) and they often times love it like crack. You can buy it at good aquarium shops.
For now, watch your fish as long as you can and see what his actions are, if he's breathing then it might be fine to wait a little longer. I assume he is breathing tho.