I do clean by a carbon brush once may be before playing but M97XE seems too finicky
Okay, just to be more clear --- clean your old records with one of the methods recommended here on this forum. Look up "Manav Malvai Zero Paisa Cleaning Method". It's effective. A vacuum suction after the said process will give you very clean records. Instantly put the cleaned record in fresh inner sleeve. If dust is too much bother use Ziploc type outer cover made by our member Rikhav to get complete seal from dust and moisture. The above is primary cleaning.
Brushing records before each play is ONLY for dusting off lint and dust that gets accumulated during handling and storage. This is secondary cleaning and is not a substitute for the primary cleaning.
Further, as already suggested by G401fan, use the brush of your M97xE. Besides clearing the path for the stylus, it will also help it track warped records way better than not using it.