Framing is rarely square, plumb, and straight enough for doors.

I always start on the hinge side of the jamb.

It needs to be tight on the stud at every hinge location, by luck or shim.

Make sure you use shims in pairs from opposite sides to keep the jamb square in every direction.

Ideally, you shim behind each hinge (usually three nowadays) and three places on the latch side.
Occasionally due to having to hold a jamb tight to one side or the other, you may end up using fewer shims on a side.
Casey

Thank you all for sharing your thoughts. On almost all of our doors, there are only shims on the hinge side. The trim is not on yet, so I guess I need to decide if it is worth going after with our contractor.