There is normally a brick moulding trim on a door frame in a brick wall. If you add one it would cover up much of the paint and provide a tighter gap.

As macv said, installing brickmold would solve a lot of your problems. Getting paint off of brick is never easy because brick is porous and some of the paint ends up in the pores of the brick. You can try a chemical cleaner like Goof Off or just xylene (the main ingredient in Goof Off). It's nasty stuff to read the precautions carefully.

If you want to get gung ho, you can scribe the brickmold to the brick to minimize any gaps.