Yes, I remove all that stuff around the roots of the plant. Be careful to not damage more roots than you have to.
My Tank: 20 Gallon Freshwater High Tech Planted
My Plants: Dwarf Sagittaria, Riccia, Crypt Lutea, Rotala Macandra, Corkscrew Val, Blyxa Japonica, Ambulia, Anubias Nana, Flame Moss, Christmas Moss, Echinodorus Parviflorus.
My Fish: 1 German Gold Ram, 3 Praecox Rainbows, 1 Otocinclus Catfish, 1 Siamese Algae Eater, 1 Rummynose Tetra, 4 Cherry Barbs
My Inverts: Ramshorn Snails, Apple Snails.