You're referring to the 4 vertical ones in the red holder ? Since the sponge is primarily the mechanical side of the system I would say probably not, but agree to only replace the two worst ones first then the other two later. You could do all four at once but just watch your parameters closely for a few weeks and if anything is out of place extra WC
Curious as to how long you've been running that 406. I have the same and mine are about 2 years old at this point but in great shape.
Sent from my iPhone using Aquarium advice.