If you could wave a magic wand, what would you have happen? Would you have the built-in disappear and have smooth wall appear? Or will the built-in be functional for you in some way?

It seems it was built to hold the larger TVs of the last decade. Is your TV (assuming flat screen) going to be on this wall? Do the components need a home? Do you want storage for firewood? For toys? Dog crate?

I think you have lots of options, depending on how much work you want to do and how much you want to spend. You could pull off all that stone and drywall over it and pretend it was never there (using the smaller pulled-off stones to fix the area to the top right of the FP where you're going to have to take out some wider stones to make this work). As a part of this you could turn the space into a closet for the room on the other side of the wall.

You could simply make this opening a closet, installing a door that matches the other doors in your house. You could put a new false back in each shelf area so the "cubbies" aren't so cavernous and would be better for display or for books. You could pull out the thick shelves and install more thinner shelves for books or display.