If the windows are small, I would trim out with wood up to the concrete edge where the window cut-out meets the vertical interior concrete surface, which will maximize the light through the small windows.

If you have large windows, use some foam board under the trim.

I installed glass blocks in my windows, so I had only the width on the joints around the glass to work with. I just plastered over the concrete block and painted it the same as the surrounding sheetrock.

I've been finishing off basement windows with wooden jambs and casings for 20 plus years with no problems. If you can, you might fill in the space bewtween the jambs and concrete with low or no expansion foam to ensure a tight air block.