We have a number of homes here in Vermont that are off the grid, but they all use solar for electricity, and wood for heat and cooking.

Also there are kerosene powered appliances like refrigerators, etc. The Amish use these.

Below is the link to a company the Amish use.

https://www.lehmans.com/

A generator is the cheapest short term answer and the most expensive long term answer. How will you store enough fuel for the generator to keep running for more than a few hours? How much will that fuel cost?

I have linked to maps showing the viability of both solar and wind power all over the US. Both of these, particularly solar, are becoming much less expensive. Both would require a battery system for cloudy/windless days.

If you live in a heavily wooded area an outdoor woodburning furnace is an excellent heating option, as it can be hooked up to the existing forced air distribution system in your home.

ChristopherH mentioned the Amish kerosene appliances. I am not familiar with those - the Amish near me use propane powered refrigerators and stoves.

I know there is a natural gas powered generator available. It can be adapted to use propane. Is your home in an area where propane deliveries are available, and can you afford the propane?
http://www.amazon.com/Generac-5882-Air-Cooled-Generator-Compliant/dp/B003IWYRWO/ref=sr_1_3?ie=UTF8&qid=1381430052&sr=8-3&keywords=Natural+Gas+Generators

Here is a link that might be useful: Wind and Solar Maps

The older appliances I'm familiar with are kerosene. The newer ones are propane.

And if anybody knows about living off the grid, it's the Amish.