When I had mine they flared at any fish they could see. As for stressing them, I have read that when they see another male and think there is competition then they eat too much and this could be the reason we see some bloated fish on here. I know I lost one to bloating/constipation even though he ate peas all the time. They really don't like any situation where they feel like they have to compete, mine never did anyway. The one I put in my community tank was fine until the baby swords got as big as him then he got bloated and tried to eat everything that was put in the tank. If the tank mates are small enough and he doesn't feel like he has to try and establish dominance then I think they are fine. I won't keep them with community fish larger than tetras anymore.
Is really no wonder that most are bred in individual containers and they aren't even left with females for very long, even for breeding.