Some of the difficulty arises when people take "cold water fish" literally and begin their tank without a heater. Unheated tanks can take much longer to cycle than heated ones. Folks get tired of waiting, start adding fish and move on to self-created problem #2.
Which is - filling up that tank with goldfish. Every square inch of space filled with glimmering gold. Which, of course, they know from their reading is courting disaster. However, just take a gander at the posts on this forum alone - someone stops by, talks about their 20 gallon tank with 6 goldfish in it, gets replies from members explaining once again that stocking a goldfish tank is much different than stocking a tropical one. But, the OP argues, MY tank has clear clean water in it...and away we go. Some of those same folks ask why their goldfish have bouyancy issues, and tell us about the cheap flakes they are feeding. We reply with the alternatives, which include fresh veggies and fruits, and again, on very frequent occasions, the OP argues - too much time, too expensive.
That gets to jlk's points. Healthy water and a proper diet. Neither in the control of the fish.
I think the notion that goldfish are harder to keep comes from fishkeepers who fail to make the commitment to the way the fish needs to be cared for.