"Freedom of movement" is a libertarian virtue in any location which is not privately owned or where the owner does not opt to restrict movement. Moreover, just because libertarians advocate a fully privatized society, it does not necessarily follow that every square inch of ground will be privately owned nor that every property owner will choose to deny access to visitors and travelers.
There are millions of acres of desert and mountains that may never be homesteaded for practical reasons and thus will remain open to occasional and temporary use by anyone who wishes to travel through them or even to hike or camp there. Private property owners may create parks, wilderness areas, wildlife preserves, or any number of other areas which may not restrict entry or movement. Private roads will likely allow for some uncompensated use—especially in areas where no other roads or trails exist. To do otherwise would be to invite widespread condemnation.
The assumption that universal respect for private property would create a society in which human movement was heavily restricted is not based on reality, but on the fear (or anticipation) of an extreme version of events in which bigotry and prejudice were a society's primary motivating factors. Fortunately, these evils are almost always outweighed by the desires for peace and prosperity and by humanitarian impulses.
It should also be noted that—even in today's society—most owners of business property are happy to have customers or clients regardless of their birthplace or immigration status, and thus restrictions on human movement violate the rights not only of those facing state discrimination but also of those employers, landlords, business owners, and others who welcome their presence.
Despite the incessant complaining about what immigrants might do to 'muh property,' there are essentially no examples of immigrants or refugees invading private property once admitted to the US. They are not being delivered onto private property against the owner's wishes. They are not being forcibly quartered in people's spare bedrooms like British troops in colonial times.
The complaints against immigrants and refugees generally flow from those who happen to encounter them in stores or other places WHERE THEY ARE WELCOME but where the xenophobes consider their presence some kind of insult to their imagined heritage. There is no right to be surrounded only with people who look like you once you leave your own property. If you don't wish to be a hermit, it's advisable to learn some tolerance.
The ever-tightening restrictions on immigration are not based on any real threat or danger (either to individuals or to the state, despite its claims), but on bigotry and prejudice aimed at particular races and religions. That some self-styled libertarians are expressing support for such restrictions—and even attempting to argue that they are consistent with libertarian ethics—is as absurd as it is revelatory of the underlying intolerance which has shaped their worldview.