I'd take the shrimp out - it served its purpose. All it will do now is continue to pollute your tank with nitrates. Since you did measure nitrates, then yeah... a water change is in order to reduce the nitrates as much as possible before adding more critters. Two or three 25% PWCs should bring them down pretty good.
[Edit: thincat snuck another line in there after the bell! Regarding leaving it in or out, if you don't plan on adding any fish for several weeks, then yeah - like thincat mentioned, leaving it in will help keep the bacteria "fed". Nitrifying bacteria, if left without an ammonia source (like fish poop, fish respiration, excess food, eetc), will die off, leaving your once cycled tank now back at square one. But if you're going to add fish right away, no need to leave the shrimp in.]