You could eliminate the latch on the pantry door and install a magnetic close at the top of the pantry door, and use cabinet hardware to open that wouldn't stick out quite so far. Or switch the hinges so the door opens the other way. Or use a bifold or double doors.

It's frustrating to have to catch the fridge door. I am having to make sure our pantry pullout cabinet to the right of the fridge is not open at all. That's something I can get used to and check before opening, but with your setup there's no getting used to it. Hope you find something that works!

I found a flush ring pull that fit in the bore for a door knob. You could combine this with a ball catch or magnet like jsweenc suggested. Of course, now I can't find on online, but they exist. The one I used had a half ring in the recess.

Here is a link that might be useful: sorta like this

http://www.myknobs.com/haf16116600.html

Try googling cabinet hardware flush ring pulls, finger pulls and lots will come up like the one above.

When I was starting this process, I thought nothing was harder than trying to find a cabinet layout that worked for all of our requirements. Now, I think it is the little things like this, going thru dozens of web pages trying to find the right last detail item...

Good Luck!! :)