In tribal times, there were no maps and the borders moved a lot, but when you crossed them, you generally got driven back or killed.
This goes back to before there were humans, and all other territorial animals do it, too.