Effectively all of your nitrifying bacteria should be in your filtration, not the substrate or decor. I have swapped substrate multiple times with absolutely no issues. I have also moved the fish and filtration (and nothing else) from an established tank to a completely new setup with no issues at all.