The key at the end of your cycle is to do a big enough pwc to get nitrAtes below 20. If you've got the heater cranked up for the cycle, remember to turn it down.
Switching substrates is likely to cause a mini-cycle, but as long as you keep a sharp eye on your levels and are prepared to do pwc's as necessary...it won't be the end of the world. Right after your cycle ends is a delicate balance point, so it's not the best time to make changes...but if you have the option of switching the gravel, then testing your BB with the lure ammo a few times you should be good to go. Leaving some of the gravel in the yank will be a help, but I wouldn't believe having it in a bag with holes is going to be as efficient as it was.
Did you cycle at 4ppm? If so, losing the BB in the substrate shouldn't have a major impact on your ability to keep the tank stable. 4ppm is designed to be overkill and grow enough nitrifying bacteria to handle a huge bio-load all at first.
Personally I'd hold off on fish for a few more days, switch the substrate, continue to dose the ammonia to ~3ppm and make sure it can be dropped within 24 hours.
Check out the guide and FAQ in my signature for a bit more info
*Same time, good answer Jose