I'd hold off on bringing in the mold experts, Mold spores are naturally in the air everywhere and remediation when they find them can run to thousands of dollars. maybe as a last resort..... I think you are on the right track regarding drying out the basement and getting a good professional cleaning. Are you re grading the outside of the house, making sure gutters are clear and directing runoff to make water go away from the house? Running a dehumidifier or two?Taking other measures to make sure the basement stays dry? If so you might get permanent relief from the lurking odors soon. Something you may not have thought of is drains and how they are vented. Sometimes odors such as you describe can come from drains. If you have standing drains in the floor or rooms with drains you use infrequently you may be able to abate some of the smells by pouring water down them once in a while. The residual moisture in traps can dry out leaving the drains open and 'breathing' . best wishes

Thank you so much for your comment. Excellent advice. We have since discovered to our amazement that the problem in the rear rooms on the second floor is caused by moisture getting behind the exterior vinyl siding. The problem in the middle rooms on the second floor appears to be nasty basement air traveling via the channels where our electrical wires run. We are trying to do what you suggest re drying out the basement and running a dehumidifier.

were you able to resolve the smell? i am currnetly expericeing this earthy musty smell throught my 1st story. The home did endure some heavy rains when constructed . We have staining on joists as well. Mold spore air test had comeback negative. Was curious if it was your joists and how you we able to pinpoint that . Currently fighting with the builder to remedy this issue.