Hmmm, first thing I do is take LOTS
of pics. For example, yesterday, I took 163 pics, from those there were about 10 that I felt were of the quality to share. Taking pictures is about capturing a moment, the more moments you capture, the more likely your going to capture the
Next, Tripod. Buy a tripod (they're cheap at Walmart, although a more expensive on will last longer), a tripod will steady the camera and help prevent blurriness from camera shake.
Use the auto time, again this prevents camera shake (this is, of course, only good for sessile critters).
When taking pics of moving objects, there is no way around it, you gotta be using a faster shutter speed, this means you either use the lash, or you fiddle with the exposure. For the pics of the clown I took (in the new aquisitions forum), I had the exposure bumped up to +2.0 EV, I had the camera stopped down to an aperture of about 3.6-4.0 (lose some depth of field, but it allows more light in) and an ISO equivelant of 200 and a shutter speed of 1/60sec. It's about capturing the light, slower shutter speeds allow you to have a larger aperture, which increases your DOF, but it makes it pretty hard to prevent motion blur.
Use manual fous whenever possible, I can't tell you how many times I've had the perfect pic ruined by not using manual focus, something (other than the intended subject) was a better target for the cam and it stole the focus.
If your cam has it, use the macro mode. I sue it for all my aquarium shots, no matter what the distance, the button is usually marked with a flower or a bug and on my display, when the camera is in that macro sweet spot, the flower on the display turns yellow, if at all possible, take your pics in that range.
Know your cams limitations and work within them. If your cam has a macro focal distance of 5", don't try to take one at 3", just stands to reason, the cam will take better pics if your working within it's limitation.
Did I mention take lots of pics? While your trying to figure out what settings take the best pics....keep a log of the pic number and what the cam was set on, that way you can review it later (if you have photoshop, this info is stored in the file in the cam, and you can pull it up easily and there is no need to keep a log, some editing software doesn't access this info, but it is stored in the jpeg file).
Other than that, I'm not to sure what I can tell you. There is definitely a benefit to having a higher quality cam, but I have seen some lower end cams take great shots, cause the person using it knew what they were doing. What cam do you have? How about posting a pic, so maybe we can see where there might be a problem...