yes. That is a concern. You have to have a plan to dose it correctly and measure your CO2 levels, especially initially as you set it up.
One con I forgot... unstable CO2 that is higher than equilibrium with air, but lower than say 20ppm or so can actually contribute to algae growth rather than help prevent it. So too little is also bad... but on to your question about too much:
A pressurized system will generally come with a selenoid valve. This is an electric valve that will control whether or not your co2 is flowing. There are basically 2 ways to automate the process and keep from adding too much CO2:
Timer: Most will set the selenoid to open about 30 minutes before the lights pop on, and turn off 30 mins to an hour before the lights go out. Plants only use CO2 when the lights are on (during photosynthesis). Adding CO2 at night can cause the tank to reach dangerous levels
pH Controller:
This is the best way to control your CO2 (although more pricey). You set the controller at a specific pH and plug your solenoid into the controller. Since addition of CO2 lowers pH, the controller will turn off when the deadband of the controller hits the number you set. It will turn back on when the pH rises back above the deadband for your setting. With this method, the CO2 level in your tank is essentially constant. I use this method, and have literally not touched my CO2 system in over a year.
There are a couple common ways to measure how much CO2 you have in the tank: either by testing KH and pH and extrapolating on a chart:
Or by using a drop checker, which gives you an accurate (if not exact) measurement of the CO2 level in your tank.
Also, as to the original question, it can kill your fish if levels get too high, and there are horror stories of selenoids getting stuck open and killing all the fish. But this is rare. And it take a lot of CO2 for this to happen. I have heard of people keeping tanks at 50+ppm with no reported effects on fish (recommended is right around 30ppm CO2).