If you have cracks that water can get into, there is a risk that it will freeze and cause the cracks to get larger and larger because water expands when it freezes. Eventually that might result in a bigger problem. That chimney sweep sounds like he knows what he's talking about, but I do not think anyone in this forum can give you a complete answer without looking at your situation. I would go to another chimney guy -- either another sweeper or a contractor who specializes in chimneys -- and ask for a second opinion. In any event, you should get several bids for the work, and ask around to find out who is really competent and reliable. That should be easy if you have neighbors with similar chimneys.