I am a bit confused, the pump "sits on the glass" and is an "inner filter pump." Is this an outside power filter? A box on outside of aquarium, hanging over top of aquarium? You can lessen sound of box rattling and vibrating against glass with a thin foam or sponge pad between filter and glass. Pet shops sell such things, but you can use thin kitchen sponge or foam packing sheets that are wrapped around electronics and other stuff you buy, or thin bubble wrap.
Also check the impeller and its compartment for debris, like a small piece of gravel.
But since you just got an old aquarium, is the pump old too? The bearings and gears do wear out over time, . Worn bearings could cause the impeller to wobble and hit against sides. Worn bearings and gears just make more noise than new ones. Depending on brand there may be rebuild kits and replacement parts available.
If none of that, or a new pump don't help, you could always do what my son's did when there cars started making a noise, turn up the stereo or wear ear plugs!