I suggest you hire a competent chimney sweep to clean the chimney. (You should do this anyway unless the seller has proof of having done that recently.) And you can ask the chimney sweep about your remodeling ideas. He should be able to tell you what you can safely do.

The general principle is: You need a certain distance between the firebox and combustible material, which is called the "clearance." Something within the clearance must be noncombustible, like tile, rock, metal, etc. because it is likely to get very hot and might catch fire if it's combustible. So the trick is to figure out what the clearances are for your particular chimney. For prefab units, the manufacturer usually tells you what they are so the installer can do a safe installation. If yours is not a prefab, you need to get a "reasonable" estimate from a competent chimney guy.

If you do not like that whole stone wall look, I am sure that much of it can probably be replaced with drywall, just so long as you observe the clearance limits, whatever they are. If you cannot get a straight answer about your clearances from a chimney sweep (they may not all be knowledgeable, and most are not "certified") you might go to the website hearth.com and post your question there. The guys on that website just love to answer questions about fireplaces. It may be possible to figure out the clearances by lighting a fire and feeling around the firebox, or maybe there's a laser device that reads the temperatue of surfaces. The guys in hearth will tell you.