Fist of all, an air stone should not be placed directly under a filter as you found out..
Also, yes, the air stone is too big..
an air stone for a 20 gal
tank only needs to be 1-2 inches in length.
Many people over kill the air stone length.for smaller tanks
You just need something to cause some surface agitation and this is what actually adds oxygen to a tank
An air stone also helps circulate the water which is also very important .
Granted, an air stone add some interests to a tank but this is a side benefit..