Yep. you most likely have a small beneficial bacteria base, so a tiny bit of a bioload will be all it can take. But as you add things, slowly, the base will grow and eventually you will have a strong colony of beneficial bacteria.
I would add your inverts first, couple at a time...if you add too many or feed too much you will get an ammonia spike. I have started tanks with nothing more than live rock and added fish every few weeks and never saw any spikes. But also keep in mind that is in a larger tank so a smaller tank like yours will show a larger effect with a smaller amount of "waste". Thats why they say a bigger tank is better, spikes take longer to happen cause of the volume of water.