people vs workers
People aren't just workers
Some libertarians (or those who lean in that direction) support a free movement in not only goods, but people. They think that if we allow refrigerators from other countries to come here freely, that means we should allow people to come here freely too.
Some of the same and others tend to focus only on someone's fiscal benefit to the U.S., treating people as just a factor of production.
Both those are wrong: people aren't refrigerators. A refrigerator just sits there, but people do things beyond just working. Some write symphonies, some commit crimes. Some have a low fiscal benefit to society but - due to their other actions - have a high overall benefit. And, vice versa.
And, unlike refrigerators, people grow up and then many years later age. Refrigerators can just be thrown out when they don't work anymore, but even most libertarians will probably admit the same isn't true of people.
Refrigerators don't have an impact on the U.S.'s politics, but people do: either through voting or through just forming a power base for politicians and in some cases foreign governments inside the U.S.
The political impact of one group of immigrants has a cost to some set of people, and a benefit to another. But, you'll be hard pressed to find a libertarian-leaning promoter of the "people are just refrigerators" school of thought acknowledging that.
If you see a libertarian-leaner treating people as just workers, ask them to put a price tag on that same person based on all their actions, not just their current working ability.