I don't know about protecting the outside, but are the pipes accessible such that you could have a shutoff valve installed inside the house that controls the water to the hose bib?

An inside shutoff is one option. Otherwise, you could buy a lock for it. The link below is an example that I found from googling.

But I think I'd talk to your neighbour and tell them that it isn't acceptable for them to use your water or modify your property in any way!

Here is a link that might be useful: faucet lock

Think the Amazon product is only meant to keep out "kids". Whoever did this actually took the cap off the totally unused pipe that was not operational and put his own hose bib on and caulked it and everything. Now all you have to do is turn the handle. looks like the hose bib could be removed (using a hefty plumbing wrench which apparently this guy has), and handle and all removed, and replaced with a new unlocked one. What keeps one of those locked faucets from being replaced if the guy has plumbing tools/equipment? and option to work undercover of the dark?

I guess an indoor shutoff is the only thing. I will be seeing my plumber soon and will talk to them then. Thanks.

At any point you can add a ball valve with a padlock near or at the house. Be sure to send the neighbor a bill for the parts and labor. Registered mail, of course.

And check your electric bill if you have outlets on that side of the house. What a brazen thief! I guess you live in an area where water is expensive, otherwise why would someone do that? Good luck with resolving this.

Wait ... the guy removed the cap on the end of the pipe and replaced it with a hose bib?!? Wouldn't the water be spewing out of the pipe when he took the cap off?
I thought that there was a hose bib already on the pipe, but the handle was removed and your neighbour simply added a handle.

An inside shut off is really your only option.

Do you live alone? If not, are you absolutely certain your partner/spouse/roommate didn't install the hose bib, or perhaps you had a plumber do it during another job and forgot about it? As previously noted, removing a cap and installing a hose bib would be extremely difficult without access to your water shutoff.

the best method is to put a shut-off before the hose bib inside the house. If that is difficult for you to do or costly, I'd shut off the water main, remove the hose bib, cap it and see if your neighbor decides to re-install the bib. I would hope he/she would be able to put 2 & 2 together on this and not reinstall the bib. If so, you can let by-gones be by-gones. If it is re-installed without discussion with you I would have a very different discussion;) Your neighbor may simply be using your water to wash the car...or you may be paying for the irrigation and house water. Check the water bill for consumption.

Thanks for the extra comments. I'm beginning to believe that a contractor who did extensive work on our house actually did this. It was a few months ago when he last worked here. I contacted him and asked him about it and he denied but I've learned his memory is not always that great.
Given the information from aliceinwonderland, and tip from jackfre, I think we will recap it and pay closer attention and see if a hose bib is re-installed.
THANKS VERY MUCH for everyone's comments. I'll let you know if a new hosebib is re-installed.

Just remove the handle or buy one of those locks. Don't bother removing the whole hose bib and recapping it. I think it's VERY unlikely that your neighbour did this.

If he pulled off the end cap and tried to install the bib and faucet with full household water pressure that would have been a disaster - water everywhere and probably interior damage from water.

I very much doubt a neighbor did this. Very strange story. Personally if I had a neighbor that I thought would be this brazen I'd be installing security cameras to the outside of my home.