The general rule that I've read is that full grown goldfish need *at least* 10 gallons of tank space a piece. That is only if they are Fancy goldfish that reach a size of only 6-8 inches. Common goldfish or comets get larger and need even more room. Goldfish are notorious waste producers and they prefer water that is cooler than what Angelfish need. I don't know how big your tank is, but even if it is big enough for all the goldfish, I wouldn't keep the angelfish in there with them...
I used to keep 3 full grown fancies in a 29
gal tank, and that was definitely the limit for those little eating machines! Anything smaller and they wouldn't have grown or spawned.