Bettas breathe from the surface through their labyrinth organ, this is why they are a good candidate for filterless tanks, because they really don't need the surface agitation. Also bettas should be fed small portions of pellets or meaty foods, leaving far less chance for uneaten food to be left behind to rot and cause a rising ammonia level. A mature 5 gal tank w/ a single betta should be able to maintain zero ammo and nitrites even w/o a filter since the bioload is small and there should be enough BB present on the tank walls, substrate, and decor to manage it. You should test often though, especially the first month, and make sure the parameters are staying where they need to be. So, if you don't have a test kit, I wouldn't even think about taking it out at this time. Live plants are also a really good idea if you're going filterless. Also be aware that w/o a filter it's likely you'll end up with a build up of protiens at the surface that resembles oil. Personally I wouldn't remove the filter if it were me, but it can be done.