As above. Gravel vacs should get it fairly clean but it's never going to be great.
In a larger tank, something like loaches are great for turning over the gravel a bit so debris can be swept off to the filter.
When I first kept fish several decades ago, I used to do that with a complete tank, filter and gravel clean. Surprised any fish survived really. I always used to have a few deaths doing that (although amazingly most survived).
I had the same problem in a 6 gal
tank and gave up on the gravel. Swapped the tank over to bare bottom. I get green spot algae on the base now so it sort of looks like a meadow.
I put more biomedia into the filter and then several weeks later took out most of the gravel. Didn't clean the filter or tank glass so as not to disturb remaining bacteria. Then over a few weeks , the rest of the gravel went. Also I used prime as a water conditioner just in case.