"does that include the inefficiency off all the cold air being dumped into the house, and all the warm air escaping out of the chimney? "

No, it does not. That makes it even more economic to change.

Just converted to hi eff condensing gas boiler condensing, 94%. I can speak from personal experience, you will see a difference as far as cost goes. I replaced an 85% eff. oil boiler that was five years old. My basement was a constant 78, I don't live in the basement , waste. Combustion air as well as exhaust air piped thru PVC pipe out of the side of the house. An outside thermostat is also used to modulate the boiler temp, you don't need full boiler temperature when its 45 outside. As far as your kitchen being cold it won't help that, there is very little residual heat that emanates from these units. Your kitchen may not have enough heating element or there are serious drafts that need plugging.
This is not a cheap undertaking , although there are rebates available. Lastly this is not a diy job. You are dealing with gas and toxic fumes. I've got a enough experience to know what should be done professionally.
Before you do anything get an energy audit, this will give some insight to what direction to go. Most states have organizations who will do this for free.