It's hard to believe it's insulated, or even lined, if air is coming out from between the stones. I suppose it could be insulated and lined above the fireplace but not below. If that's the case, the cold air is probably coming from the cellar. The first thing to do is to check the cleanout door at the chimney base. It may be open or it may not form a good seal. Before embarking on a rebuild, I'd certainly have the whole thing checked by someone else - a mason or certified chimney sweep. If it turns out you have no liner and/or insulation it would be far easier and cheaper to line it with either a stainless steel liner or a poured in place system such as Supaflu.

Thanks for the information. I do not have a cleanout door at the chimney base. But what you are saying makes sense. We live in an area with limited fireplace stores I've had both of them out here and still no nothing. I will now check with a mason and see what they tell me. Thanks!