Worries include the loss of Canadian visitors and the impact anti-immigration efforts will have on the work force.