I know there is a "setting tool" that works like a pop-rivet gun to expand the molly without rotational force. I've never heard of a special "wrench".

Can you post a picture of your wrench?

Sure, I'll post it later. I probably should call it a tool instead. It basically holds against the flange teeth while you first tighten it so the whole thing doesn't spin.

The wrench used to (ad sometimes still does) come if you purchase boxes of 100 bolts at a time.

The setting tool is far better and usually under $15.

One pull of the lever and the bolt is expanded in place.

Probably too late, but you don't have to use the keyhole hanging slots. You could use a French cleat.

Here is a link that might be useful: Hangman french cleat

I considered a french cleat, but it wouldn't work in this case. The mirror frame was concave due to leaning on the wall over a mantel for years (and I didn't really want to drill into the old wood).

Basically I extended the molly screws out about 3/8" inch to reach the keyholes to make it appear to hang plumb overall. This was tricky also because I had an interior designer of 35 yrs experience in the room at the time.