I know exactly what you are talking about. I explore New Mexico a lot and the closer I get to the border the higher the anxiety. My pet fear is being in a tent with no view of what is around me while any one who comes up can see where I am and all my exits. I think the odds of some six o'clock news event happening is small but it still makes for something other than a peaceful night's sleep.
Because of this paranoia I built some special motion sensors that I place in the camp area and on the trail or road leading to my site. The sensors contact me by radio if anything over 30 lbs crosses in front of it. Depending on the terrain, I can locate the remote sensors up to a couple of miles away for a good "heads up" when someone is coming. The real value is the peace of mind. When I wake up in the night and think I heard something I'm not afraid to stick my head out of the tent for fear my head might be lopped off with a machete. If someone is out there this system will let me know. So far there has never been an alarm except when I step outside to relieve myself (that also tests it!).
Just wanted to tell this story so you know that you are not alone in your paranoia!