You can leave the food, but you'd best do a large water change to remove the food that has already dissolved in the water. I'm a Goldfish keeper too and these fish require very large, frequent water changes. By just changing most of the water, you'll remove any pollutants from the food that has dissolved and all the new water will dilute whatever the fry don't eat. This way, there's little or no waste.
"Fear not, my young apprentice. Just change the tank water."