PVC is generally inert unless it's heated (by heated I mean burning or otherwise damaging it with heat). If you burn PVC it will release poisonous and toxic gasses. So in an aquarium setting the PVC should remain inert.
It is used in plumbing. Most houses built these days have PVC pipes. Some people use PVC as cheap decorations and caves and whatnot. Some people use PVC for plumbing their tank and sump or refugium together.
Short answer on why is that it's considered safe and cheap. Beyond that it depends on what they're using for as to why they're using it.