It would help to know what machine you have. It's obviously something in the way your detergent tray is designed. I can't see how vinegar would leak but fabric softener would not? I use vinegar in my Miele W4842 no problem.

Some FS dispensers work on a siphon principle. A small amount of softener sits in the dispenser, but when filled to the brim with water a siphon action is initiated through a small tube. This may also happen if you fill the dispenser to the brim with vinegar. My suggestion is not to overfill the dispenser. If you feel that you can't get enough vinegar in there, I would suggest switching to citric acid instead. Just dissolve 1 TBSP of citric acid into a FS-sized shot of vinegar. This should be enough to provide a sour final rinse.

Yep...if you fill it to high it can siphon early. With my drawer if you push it in a little to fast it causes it to siphon too soon as well.