I'd say the answer is a resounding... "maybe."
If you're having water quality issues (cyano, high nitrates, algae, etc) then I'd say the answer is "yes." If your water quality if fine and the fish seem healthy, then I'd say the answer is "no". Probably not the answer you're looking for, but there really never seems to be many exact answers in this hobby! Could you cut back on your feeding with no issues? Most likely.
One thing I'll suggest though is to switch away from the brine shrimp as a main course and mix it up a little - some mysis shrimp, some of the meaty seafood mixes (Rod's Food, etc) and other varieties of meaty foods. Brine shrimp - from what I've read - don't have quite the nutritional value that other foods do. That's the main reason they gut-load the shrimp with vitamins, spirulina, etc... so the fish are at least getting *some* nutrients from it. Someone here once equated feeding your fish brine shrimp to us having popcorn for dinner every night.