Not an expert here, but I would guess you would at a minimum need to remove the pipe and fill the hole with refractory mortar. A local chimney company should be able to give you a definitive answer and be able to do the work in a way that would satisfy your local inspector. You'll want them to inspect your chimney before doing anything, and this may turn up other problems if it hasn't been used or maintained for many years.

While you're considering this work, you really should think about putting in a wood burning insert instead of burning directly in the fireplace. The insert will burn much more efficiently and will give you far more heat than burning directly in the fireplace. We just put an insert into our fireplace and it's really made a huge difference.

I definitely agree about the insert.

Fireplaces have a "negative efficiency" which means they will take heat out of the home and up the chimney. That's probably why the unit was converted.

Today's modern inserts can be damped down to a point where you can see a lazy flame which can be very relaxing. They're very efficient too. And clean burning.