For freshwater tanks, the need for water changes is multi folded. For starters, it helps reduce nitrate levels continuously so that there shouldn't be a nitrate accumulation at all for the fish to contend with. I do weekly water changes of only 10% -15% and stopped regularly checking nitrates in my tanks about 30 years ago. They weren't an issue then and I haven't changed my fish keeping practices. ( FYI, just today I did do a nitrate test, for kicks, in a tank that has been an experiment for a while with having some live plants in there for nitrate control and the level with no water change for approx. 6 weeks was under 5 ppm.)
Another reason to change water is to drain off any sludge that may be developing in the gravel or sand bed. Another is to help add back any minerals that the fish have absorbed out of the water. ( We used to call this "sweetening the water".) There are some other more technical reasons as well which I can't remember atm.
For marine tanks, again, reducing the accumulation of nitrates is a main reason for water changes. In today's hobby, there are supplements available to add that water changes used to be done for. Pulling out extra proteins that a skimmer may not have gotten to yet would also be accomplished by a water change.
I was told, long ago, that "If your tank's inhabitants look better after a water change, you should be changing water more often because the setup is either lacking something they need or something is accumulating that they don't need. "Figure out which and fix it!!" These were words my fish lived by.
Keep in mind that it seems that today's hobby seems to be more about how much stuff we can get into a tank with the least amount of followup work. Unfortunately, the more "stuff" you have in a tank, the more work you should be doing. Understocking and overfiltering (within reason) will help reduce the frequency of needed water changes. I doubt many tanks fit that scenario.
As previously stated, with a proper water change system, it shouldn't take long to do water changes in your tanks. I did 10 tanks today in just over 2 hours because 2 of the tanks were complete strip downs and I moved fish around so they needed to be acclimated. Usually, that would take less than an hour of my time.
Hope this helps.