The best way to tell is males will have red on their noses and more red on their fins.
You saw 40min worth, chances are they've gone at it a few times before. I'd see the the same two go at it over and over in spurts. And yeah when TB's go at it, everyone stays out of their way. It's Hilarious, my RTS IS the top dog of the tank, but it wanted no part in the fracas that was taking place today. Where ever my two TB's were going at it, my RTS would go to the opposite end. I've seen other oblivious barbs literally swim into fights and just got twirled around and around.
The main thing to look out for is if one feels so stressed after it loses it hides all the time.
I had this one male Albino who was the smallest barb and for the first year it went toe to toe with every male. It would lose, but it still had that swagger. Then after one of it's losing bouts, I noticed it just hid under DW all the time. After a while I noticed it's fins were getting nipped. I eventually had to put it down as it was just bullied to the point it was a wreck.
Sucks but it goes with the territory on how TB's establish their Hierarchy. Low man on the totem pole, usually the smallest TB ,is perceived as the weakest thus bullied.