The only thing I can think of is that it might be harder to keep gunk free... Plus it wouldn't function as efficiently as biomedia because there's lower water circulation rates going through it compared to if it was in a filter with a filter pump constantly pushing water through it. You could use a UG filter to help preserve the water flow rates though.
I'd avoid doing that *— for the aforementioned reasons. Increased maintenance and a lesser efficiency rate are two very persuasive reasons to avoid that approach.
Just my $0.02.
Seachem matrix is nothing but seriously overpriced pumice. You can buy pumice gravel on eBay for a fraction of the price.
It would probably be amazing for a under gravel filter because of its nitrate reducing capabilities coupled with the low flow of a ugf.