If he does good work at a price you can afford, then MAYBE. But, if you do, make absolutely sure your contract with him contains an explicit "time is of the essence" clause and a "liquidated damages" clause that requires him to pay you a per diem amount that is sufficient to pay your reasonable living expenses (decent hotel/motel, restaurant meals, extra travel expenses, etc) and the cost of storing your furniture if the new house is not completed before you have to move out of your current residence. (Don't get greedy in determining the amount at which to set the liquidated damages b/c while courts will enforce reasonable liquidated damages - enough to offset amounts reasonably likely to be suffered by the non-breaching party - they don't like liquidated damages that are designed to penalize the breaching party. If the contractor agrees to a contract containing a time is of the essence clause and liquidated damages, then he is going to be a whole lot more focused on getting your job done on time! Just be sure that you are ALWAYS ready to respond to any requests he has for decisions and that your building documents (including specifications) are as complete as humanly possible so that he never has any reason to blame you for delays. Also, set the deadline at least a month or so in advance of the day you need to move out of your current residence and make sure the contract defines what kinds of weather conditions the builder can use as non-infringing excuses for delays. (eg., more than x inches of rain in a 24 hour period, more than x inches of snow on the ground, colder than x degrees, wild-fires within x miles of the building site, etc.) Weather WILL likely cause some delays and it is hard to blame a builder for those. So build in a bit of extra time.

Talk to attorney licensed in your jurisdiction about drafting the necessary language for you. You should have your contract reviewed by an attorney before you sign it anyway. Shouldn't cost you more than a few hundred dollars and that is chicken-feed given that you're probably entering into a contract for largest amount you've ever spent in your entire life.

Thank you so much for the advice. I will look into including that on the contact and will definitely have an attorney look it over!

This would drive me crazy. One thing I personally love about our builder is that he is very good at getting back to me quickly. Several points throughout the build I have changed my mind on something and if my builder were not readily available then I would not have had a chance to catch things in time and make the necessary changes.