We use a product called Cutter Backyard Spray (or something like that) for mosquitos, not sure if it would work for horseflys or not, but it would be worth a shot. It works really well for mosquitos for about 6 weeks. It is about ten bucks @ Wallyworld, and you hook it up to your water hose and spray the yard down.

Other than a full screen enclosure, you might need to make friends with them :)

I've read that some people use badminton racquets to swat them. (It works with wasps and aggressive bumble bees.)

If you don't have an unlimited supply of horseflies you could work out your pent up hostilities toward them and swim when you had taken the most active flies out. Just be careful if you are swimming with other people.