That's not a bad idea. Or, instead of a gravel topper, if you wanted something a bit finer you could consider 3M colorquartz. They come in a variety of colors and have a good grain size.
I think the trick is going to be going deep. You're going to need a deep substrate so that you can plant your plants really deep to make them hold better. Once the roots get established they'll do better on their own, but a deep substrate layer will help hold them while that happens, and capping with gravel of some sort should be helpful in your scenario.
In the DC Metro Area? Check out GWAPA