It is a simple fact of reality, that when you restrict air circulation, you create an atmosphere that allows mold and mildew to flourish. So now that you have remediated the affect of, ensureing that you do not repeat, should be sufficient.

One solution is to paint the walls using paint with a fungacide (basically bleach) added to it. This helps to prevent the mold/mildew from growing on the wall. Zinser makes some great paint products for remediating/preventing mold/mildew. I had very good luck with Perm-white in my bathroom.

Here is a link that might be useful: zinser perma-white

I agree with the perma-white paint. I believe it has a 5-year warranty for mold/mildew. I used it in my bathroom well over 5 years ago and still no sign of mildew, which was a major problem before I discovered the perma-white. I have also used it in my closets with the same great results. It comes in satin or semi-gloss and can even be tinted if desired. Great stuff!

You need to improve air circulation in that closet, or the mildew problem will recur on clothing. The walls are not the problem.

Adding some vents to the door, even adding a fan that runs a couple hours a day to swap out the stale air with fresh.