You should use an exterior spackle that you would float over the peeled areas and then sand smooth after it dries.

It also could be moisture. Here's a link to an article that describes common paint problems -- could be something there.

Here is a link that might be useful: common paint problems

I agree with allison, it could be a moisture problem. You can check the moisture content of the wood with a moisture meter)available at paint stores. It could be moisture migration from the bottom of the door during rain and puddling of water. It could also occur if you have a heated garage and live in a colder climate. If I were to choose a primer for this job, I would probably prime with an oil based primer. Oil based primers tend to seal wood better than latex. You can use Elmer's Wood Filler or Exterior Spackling products. It still requires a lot of sanding to get the wood smooth. Prime after all the spackling and sanding is finished. Then topcoat with the SW Duration.

SW makes an exterior spackle that might be the ticket. Use no more than necessary since it dries hard.

Prime the bottom of the door also. This will prevent moisture from wicking into the door.