If you have an older edition of Dr. Innes's Exotic Aquarium Fishes, you'll find that in the old days of metal frames and mastic tanks, pouring varnish into the corners was a way of sealing leaks. Back then it was an oil based natural varnish, spar varnish was preferred. I can see no reason a modern oil based synthetic wouldn't work as well. I wouldn't trust a water based to not deteriorate.Best to then it down to about half as thick as out of the can and let a coat sink in, dry and repeat a few times then apply a couple of regular thickness coats. Let it dry real good for a few days.
How are you fastening the pieces together? I don't know of any glue that will last underwater, except silicone. A few tight criss cross wraps, tightly knotted of brownish light string or thread might do. Multi stranded synthetic, like polyester, would probably last longer. Be sure and soak knots with varnish.
If you still worry about chemicals, soak the varnished bridge in a pail of water a few days, changing water daily.
And how are you keeping your bridge from floating off?