I guess I'll hit the normal questions that will help us help you with algae issues...
1. Have you tested your nitrates and phosphates? If so , what are those numbers?
2. Are you using RO
water or tap?
3. What type of algae are we talking about? Hair? (Long green strands.) Cyanobacteria? (Slimy red, green, or blue sheets full of bubbles) Diatoms? (Rusty brown colored dust?)
4. How old is the tank?
5. How often, and how much are you feeding?
6. What type of lighting do you have, and how long do you have it on?
Depending on the type of algae, yes... you may need to be concerned. Diatoms are a normal part of setting up a new tank. But the other types of algae feed off of excessive nutrients in your tank such as nitrates and phosphates, or excessive lighting. Prevention of algae is a matter of eliminating the fuel that's feeding it.
Two clowns and two damsels is a lot of fish for a 24g tank. What type of filtration do you have?