With, of course, a pun on pee, meaning to urinate, the jocular phrase silent like (the) ‘p’ in swimming is used when exposing a difficulty in pronunciation.
—Cf. also:
– time flies? you cannot: they go too fast;
– why is a mouse when it spins? because the higher the fewer.

For example, the following is from a portrait of the Canadian statesman Ray Hnatyshyn * (Ramon John Hnatyshyn – 1934-2002), who had just been appointed Canada’s 24^th Governor General—portrait by Randy Burton, published in the Star-Phoenix (Saskatoon, Saskatchewan, Canada) of Saturday 7^th October 1989:

“He never took seriously anything that didn’t deserve to be taken seriously, including criticism of himself,” says William Gurgulis, another grade school chum who is now a judge on the Alberta Court of Queen’s Bench.
With a name like Hnatyshyn, there was bound to be cracks about his name: “He used to say that the ‘y’ in Hnatyshyn was silent like the ‘p’ in swimming,” Gurgulis said.

(* Hnatyshyn is pronounced /nəˈtɪʃən/.)

The earliest occurrence of the phrase that I have found is from Collyer’s Eye (Chicago, Illinois, USA) of Saturday 15^th August 1925—the author uses the punning phrase to pun on the surname Shimp:

SHIMP FAILS TO OUTSMART VETS
BY N. C. FAIR

HERBERT G. SHIMP, owner of the International Stable, according to close observers has one letter too many in his name. After witnessing the insurance broker trying to outsmart some veterans at the game it is quite evident that the “H” should be silent like the “P” in swimming.
When Postillion, a four-year-old daughter of Berrildon-Dignity, beat Gibbons in the Rainbo Handicap in a driving finish Owner Shimp nicked the betting ring for nearly $50,000. He wagered $3,000 with “Hickory Slim,” the Big Bertha of the ring, at odds of 20, 8 and 4. He took $32,000 away from this book alone. He had to wager just a peanut to pull down $15,000 from Frank Chance.
After Postillion had won Owner Shimp was presented with a loving cup and surrounded by numerous friends who handed him a lot of “bull” about how smart he was. These so-called friends made it so strong that Shimp soon began to believe it himself. He forgot entirely that he owned Postillion and knew something about her condition.
There were other races later in the day and Mr Shimp—just leave out the “H”—decided that he would do a little plunging without making any inquiries. First he set in the checks on the Croissant entry of Sun Altos and Kentucky Cardinal, which finished second. Then he wagered on Midwestern, which also ran second. Then he plunged on Ramkin, which finished a “fat” fourth.
It was in the closing race of the day that Shimp decided that the time was ripe to wager on R. E. Clark. He did not take a trip to the paddock to consult with the trainer of this colt, probably for fear it would cost him something. He merely took the betting privileges and forced the price down from 10 to 1 to 5 to 1. He had all the best of the betting. But when it came to the running of the race R. E. Clark did not do so well. For future references Mr Shimp will benefit by not trying to beat the price. It is safer in the long run.

The second-earliest occurrence that I have found is from an advertisement for Riverside Mills, published in The Daily Clintonian (Clinton, Indiana, USA) of Friday 23^rd January 1953—advertisement written by Cliff Andrews, co-owner of Riverside Mills:

Lime effectively releases nitrogen but an overdose of it locks up iron and manganese which are available in all soils in very minute quantities. Without them plants can’t produce chlorophyll. I had a hard time findin’ that word in the old Websters. I thought it was spelled without that first H. It isn’t pronounced anyway. It remains silent like the p in swimming.