Our inspector failed to notice our fireplace appears to be pulling away from the outside wall.

Our house is about 30 years old. It's on my list of things to do but I was just going to get a chimney inspection to find out if there are any real problems. I think my problem sounds way bigger than yours. It wouldn't hurt to call someone qualified and find out if there are any real problems. It might not be all that bad!

We use to have a fireplace at our other place (now a rental), I'm tired of them. They're a lot of work... all that cleaning every season and buying wood, and listening to the husband complain about how hot it is.

I just finished an update in my post "Chimney Fire". The steps I took may be of help to you. Knowledge is power and things might not be as bad or expensive as concerned. Liners run about $1200 - $1500 installed. We finished up at $600 with a professional chimney guy found through Craiglist. He even sourced out and fixed a "draw" problem the upstairs fireplace has had since built.