Yeah that shouldn't be a problem. When you do add them you will most likly have to turn you tank pump off for a few hours to let the large particles of sand and mud settle. Also make sure you have a sponge or filter sock to put in there so it helps to clear the water as quickly as possible.
If you can i would add it all during a water change. Shut the pumps off, empty the section of the sump you will have as your refugium, add everything and let it settle, then top it off with water and turn the pumps back on.