Yeah these little motors can get dirty pretty quick being most things there in sit so low to the ground. Every now and then it's a good idea to do what you just did take it out and clean it.
The red stuff on the back and front of the rotor is for balancing so leave it alone.
You can but...
i have some small motors like that they are pressed on like WoodiE said. What I do is put a pinion on the shaft and give a couple of taps on a solid surface and that end cap will pop off. It takes a little bit of effort but that cap will pop off.