[Please note: This post contains language that some readers may find offensive.]

The phrase (the) wogs begin, or start, at Calais ¹ is used to express an attitude of insularity and hostility to foreigners attributed to the British, especially the English.
—Cf. also the phrase storm on Channel—Continent isolated.

¹ Calais, in northern France, on the Strait of Dover, is the nearest French port to England.

The derogatory and offensive British-English noun wog designates a non-white person. It is a shortening of golliwog, of same meaning, but is sometimes erroneously explained as an acronym, from the initial letters of various combinations such as Wily Oriental Gentleman.
—Cf. also the erroneous explanation of the derogatory British-English noun chav as an acronym.

These are the earliest occurrences of the phrase (the) wogs begin, or start, at Calais that I have found, in chronological order:

1-: From the Crowley Daily Signal (Crowley, Louisiana) of Monday 12^th May 1947:

What’s in a Name?

A report from the recent Asian Relations Conference stresses the fact that Asians don’t like to be called Asiatics—a term which seems to them to involve a racial slur. This is not a matter for non-Asians to make merry over, but to understand.
It’s hard for a “Yank,” who takes the term with good-natured indifference to understand why the Japanese should always have looked on the word “Jap” as an insult The answer lies not in the word itself, but in the unconscious attitude of patronage behind it.
The Briton does not feel that he has lost face hopelessly when the Yank refers to him as a “limey” or the Aussie calls him a “pommy.” Consequently, he refers cheerfully to the “wog” (wily Oriental gentleman) without realizing that the joke will not seem so funny to the Asian, who does not appreciate the Briton’s dig at his own insularity in his saying, “The wogs begin at Calais.”
There are other uglier terms (like “dago,” “kike,” “chink,” and “nigger”) which many good people use thoughtlessly and without intent to hurt. At the very best, all such terms stress the “otherness” of other people. If the “peopleness” is uppermost in thought instead, the nicknames we use for each other will be bonds of affection rather than barriers of prejudice.— Christian Science Monitor.

2-: From the proceedings of a debate that took place at the House of Commons on Friday 29^th July 1949—George Wigg (1900-1983), Member of Parliament for Dudley, Worcestershire, declared the following:

I have heard the hon. Gentleman speak on this and every other occasion, and as the years roll by I recognise that language such as he uses is one of the main reasons for our political difficulties in the Colonial Empire. I recently had the opportunity of talking to some Burmese gentlemen, and one of the things they said was that they never realised until they came here and met ordinary people, what the British people were like. They thought they were all haughty and arrogant. The hon. Gentleman and his Friends think they are all “wogs.” Indeed, the right hon. Member for Woodford (Mr. Churchill) thinks that the “wogs” start at Calais. If one views ​ people like the hon. Gentleman from the angle of a private soldier, one realises that to them there are black “wogs” and white “wogs.” The attitude of hon. Members opposite to the black chap is not much different from the attitude of some of them towards the private soldier, and that is why the Forces have a great sympathy with the native peoples.

3-: From an interview by Philip Toynbee of the French novelist, playwright and critic François Mauriac (1885-1970), published in The Observer (London, England) of Sunday 27^th October 1957:

Mauriac: When I look about me I find that anti-Semitism is as strong, at least in France, as it ever was. Take the example of our only great political figure Mendés-France ². He is a Jew, as you know, and this has done him an incalculable amount of harm. Nobody admits it, but what really broke him was racial feeling.
Toynbee: The same sort of thing is true in England. We have a word “wog” which we use for all coloured people—a deliberately offensive word. For some years there had been a certain uneasiness about using this word, but during the Suez affair ³ it was in constant use again.
Mauriac: Well, I rather believe that “wogs” begin at Calais.
Toynbee: I’m afraid that may be true.

² Pierre Mendès-France (1907-1982) was a French socialist statesman who served as Prime Minister of France from June 1954 to February 1955.
³ The Suez crisis was a short conflict following the nationalisation of the Suez Canal by President Nasser of Egypt in 1956; Britain and France made a military alliance with Israel to regain control of the canal, but international criticism forced the withdrawal of forces.

4-: From The Observer (London, England) of Sunday 9^th March 1958—Jocelyn Brooke used wogs begin at Calais as the title of his review of the works of the British author Herman Cyril McNeile (1888-1937), who, under the pen name of Sapper, created the fictional character of Hugh ‘Bulldog’ Drummond:

The Sapper ethos […] provides an unrivalled range of scapegoats: Jews, Dagoes (the term includes all foreigners except Germans, who are referred to as Boches), highbrows, any sort of artist—all these are anathema, and can be (and nearly always are) beaten-up or even bumped-off without scruple by the hero. (The same goes for the working class en masse, unless they happen to be ex-soldiers.) Sapper, in fact, appeals to the latent fascist in all of us: that, I suspect, is the real secret of his perennial attraction.

5-: From Neither Side Yearns for United Germany, by Graham Hovey, published in the Minneapolis Morning Tribune (Minneapolis, Minnesota) of Thursday 26^th March 1959—Graham Hovey is quoting “a British observer who has watched the chancellor at close range for 10 years”:

“Adenauer ⁴ is a Rhinelander—a left bank Rhinelander. He hates the Prussians. And in the way that the British used to say, ‘the wogs begin at Calais,’ Adenauer thinks of the Germans on the other side of the Rhine as Poles!”

⁴ The German statesman Konrad Adenauer (1876-1967) was the first Chancellor of the Federal Republic of Germany from 1949 to 1963.

6-: From an article about the International Club of Manchester, published in The Guardian (Manchester, Lancashire, England) of Thursday 12^th November 1959:

Not long ago François Mauriac, the French writer, reminded his readers that for the British “Wogs began at Calais.”

7-: From the column Table Talk, by ‘Peregrine’, published in The Observer (London, England) of Sunday 10^th April 1960—President Charles de Gaulle ⁵ came to London for a state visit in April 1960:

For the British there are really no foreigners like the French. Wogs may begin at Calais, but the entente cordiale has a strange warming effect even on Englishmen who don’t rightly know what it is. We found the crowds watching President de Gaulle last week rather moving.

⁵ The French general and statesman Charles de Gaulle (1890-1970) was the organiser of the Free French movement during the Second World War, the head of the French government from 1944 to 1946, and the President of the French Republic from 1959 to 1969.

8-: From the review of the 1961 British film The Greengage Summer, published in The Guardian (Manchester, Lancashire, England) of Monday 29^th May 1961:

A family of schoolchildren, the eldest of whom is in that delicate stage of English adolescence, poised precariously between a gawky, hockey-playing sixth-former and [an] attractive young woman, is stranded in France when their mother is taken ill. The film seems dedicated to the proposition that “Wogs begin at Calais” and the children suffer at the hands of a most unpleasant crew of French hoteliers.