Controlling immigration has always been one of the most difficult and at times the most poisonous issues in British politics. Enoch Powell caused turmoil when he raised the issue of immigration in the late 1960s and Margaret Thatcher played the immigration card when she said Britain was in danger of being “swamped” by incomers just before the 1979 election.

After the end of communism all east Europeans were able to travel freely around Europe. Hundreds of thousands of Poles came to West Europe, driving across the border to Germany or fly to Britain. In the 1990s, nearly all construction work inside London homes or work as nannies or in the health and old-age care seemed to be done by East Europeans, principally Poles.

100,000 Swedes pitched up in Britain after Sweden joined the EU in 1995. In fact, EU membership is a bit of red herring. Britain has visa free travel agreements with 174 countries. There are an estimated 200,000 Brazilians working illegally in Britain. Mrs May as Home Secretary tried to install pre-travel visas for Brazilians.  This was vetoed by David Cameron as Brazil as seen from the No 10 geo-political worldview as a useful UK ally in Latin America and at the UN.

Indeed unless Amber Rudd proposes to make all citizens of 27 European Union nations (plus Switzerland and Norway) apply to British embassies for a visa before travelling to Britain, any system of work permits, will just bring back the black or illegal labour market that existed before 2004. That is why France, Germany and other countries quietly retreated from the seven-year transition period for full labour market access after the new EU members joined in 2004. Workers came anyway and found plenty of employers who could not find local labour for construction, old-age care, tourism, restaurant and agricultural work.

There are 1.1 million Romanians in Spain, for example, but neither the press nor politicians make a fuss about them. The 30,000 British citizens of Gibraltar cause much more burn-up political emotion in Spain than 3 million immigrants.

Amber Rudd now faces a tricky decision. Although the word “immigrants” was not on the ballot paper of last June’s referendum there is little doubt it was the major issue that decided the vote. Mrs May has certainly chosen to interpret the result as an anti-immigrant decision by the people. This has been reinforced by Labour MPs like Keir Starmer, Chukka Umanna, and Stephen Kinnock who have all come up with their own plans to stop Europeans from working in Britain though not Pakistani, Bangladeshi or Pakistani immigrants whose British-based communities are central to Labour majorities.

There are two ways of slowing down or controlling immigration – external or internal. External controls are crude cold war era work and residence visas. I worked in Switzerland in the 1980s and the controls were rigoruous including a strip down health check and car boots being searched as I went from France to Switzerland to check if I had bought cheaper beef in a French supermarket.

Today, in order to keep full access to the EU Single Market, the Swiss have quietly let drop their 2014 vote to ban “mass immigration”. Instead, they have opted for internal controls, including internal identity cards and checks, and an obligation on employers to first seek to hire local workers.

Already EU immigration into the UK is slowing down as pay in devalued pounds is less attractive, a message going round  that Europeans are not welcome, and uncertainty in many sectors that Brexit means a loss of access to key EU markets  especially in the fintech sector.

Labour failed miserably to protect local workers. Tony Blair and Gordown Brown vetoed or diluted the full application of EU directives on agency or posted workers that other countries adopted precisely to reassure local workers. The EU freedom of movement rules do not apply to state employment but there are 57,000 EU employees in the NHS alone as the UK does not train enough doctors, nurses and orderlies.

Britain has no identity card system so no-one knows who is in the country and no requirement to have made contributions to social or health insurance funds before being eligible for benefits.

Again, in other countries there are stronger internal obligations and controls which do not end immigration which is always employer driven but do make the flow less dramatic.

Might it be possible for the government to copy Switzerland and focus on internal controls? Stronger protection for British workers like apprenticeship and training to work in the NHS and other sectors also cleaves with Mrs May claim she wants to support working people.

Any external controls aimed at European citizens will be seen as a hostile act and undo David Davis’s charm offensive in East European capitals ahead of Article 50 negotiations. No EU prime minister will take kindly to London saying voters from Poland or Portugal or wherever are no longer welcome in Britain. It will inevitably lead to reciprocal measures against the 2 million Brits living in the EU.

Internal controls are a smarter modern way of controlling immigration and should be examined carefully before a unilateral Home Office decision – almost certainly unworkable – turns Article 50 into a train crash.

Denis MacShane is the former Minister of Europe. In January 2015 he published Brexit: How Britain Will Leave Europe (IB Tauris). He is a senior advisor at Avisa Partners, Brussels.