Many homes in our neighborhood (central AL) have dark soffits and trim. Many used Hardi board materials. No one has had problems. We've been in our home 7 years and owned land 2 years before building.

Great to hear! Most homes here have white or off white trim but the GC said not to be concerned.

One of the reasons lighter colors are popular on houses is that they do not fade as fast as dark colors in strong sunlight. Also, the use of white on architectural detailing makes them more distinct and prominent. Brown contains some red so it sometimes fades to a soft muddy color. I would also be concerned about fungus on dark soffits in damp climates. Be sure to look at similar colors on the soffits of other houses. If the contractor has never used this color range in this manner before I would be skeptical of his advice.