The slang phrase the (little) man in the boat denotes the clitoris.

This phrase likens the clitoris, placed between the inner labia of the vulva, to a person in a boat.

These are some of the occurrences of the phrase the (little) man in the boat, in chronological order:

1-: From Slang and its Analogues Past and Present ([London]: Printed for subscribers only, 1896), by the British lexicographer John Stephen Farmer (1854-1916) and the British author William Ernest Henley (1849-1903):

The little man in the boat, subs. phr. (venery).—The clitoris. See Button ¹.

¹ The following is from the revised edition (1909) of Slang and its Analogues Past and Present:

Button-hole, subs. phr. (venery).—The female pudendum.

2-: From Sitting on a Fortune: The Prostitute in London, by Wayland Young, published in Encounter (London, England) of May 1959:

What is the whore’s own sex-life? Here are several glimpses from different girls. […]
I’m rather abnormally built. You see, my clitoris is very high—I’m not embarrassing you by talking like this? I guess everything’s a bit far apart with me. I can only come if a man realises this; but I can’t ask them, can I? They’re not there for my pleasure; I’m there for theirs. That’s how the game is. I tell you, it’s not all ninepins.
I don’t know why I like girls. It feels safer, somehow. And I’m not talking about pregnancy. I think everybody agrees now—take Kinsey ²—that the clitoris is where women get their pleasure.
The man in the boat’s my mascot. As a matter of fact it’s every woman’s mascot, though there’s not many men’ll recognise it.

² The U.S. zoologist and sex researcher Alfred Charles Kinsey (1894-1956) carried out pioneering studies into sexual behaviour by interviewing large numbers of people.

3-: From the Evening Journal (Wilmington, Delaware, USA) of Friday 24^th March 1972:

Sex? Ideas Change on Education
By Cathy Wolff

Clitoris, vagina, penis. Offensive? Disgusting? Not to the 900 students, teachers and parents who participated in a day-long sex seminar at Wilmington High School last Tuesday.
[…]
The leader [of one group said]:
“There is an organ that every girl has that has been denied us for a long time—it’s called the clitoris.”
[…]
“It’s the only organ that exists in a man or woman solely for sexual pleasure . . . It used to be thought there were two types of orgasms (for women) clitoral and vaginal. But that’s not true. There is only a clitoral orgasm. . . Sex is best for women when the clitoris is involved.”
After the morning session, while she waited in line to buy a bag lunch, one middle-aged parent said she was glad to hear about the clitoris: “We always called it the ‘little man in the boat.’”

4-: From Pages From a Cold Island (New York: Random House, Inc., 1975), by the U.S. novelist Frederick Exley (1929–1992):

Masters and Johnson ³ could not be taken seriously by decent or civilized people and I didn’t in the least understand why women held it ominous for men that these quacks had unequivocally established the clitoris as the female orgasmic organ, a Lilliputian prick. To be sure, this was ignorance on my part (having out of boredom been unable to complete my reading of Freud, I hadn’t known he’d plumped for the vaginal orgasm), but up in the cowshed of Watertown we’d known forever about the “button” or the “man in the boat” and that a man bent on giving a woman pleasure must approach this cute little bugger with a certain worldly and heady enthusiasm.

³ This refers to the Masters and Johnson sexuality research team, composed of the U.S. gynaecologist William Howell Masters (1915-2001) and the U.S. sexologist Virginia E. Johnson (born Mary Virginia Eshelman – 1925-2013).

5-: From The Courier-Journal (Louisville, Kentucky, USA) of Friday 3^rd April 1981:

Frankfort, Ky.—An aura of television drama—aided by floodlights, six TV cameras and a host of lawyers—surrounded the opening yesterday of the long-awaited hearing on sexual-harassment charges against Agriculture Commissioner Alben W. Barkley II and two other men.
In an unprecedented public session, the two women who filed the charges came face-to-face with Barkley and the two other Agriculture Department employees—Gerald Deatherage and Doug Wheeler.
The women, former employees In the department, testified before the cameras of commercial television stations and the Kentucky Educational Television network […].
[…]
There were times of nervous levity. Laughter erupted when attorney Thomas Watkins, representing Deatherage, suggested to Barbara Armstrong—the other woman bringing charges—that the phrase “the little man in the boat” might well have been a reference to Kentucky River boaters who could be seen from the main agriculture offices in the Capital Plaza Towers.
Mrs. Armstrong earlier had charged that Deatherage had persistently chided and embarrassed her for not knowing the meaning of “the little man in the boat.” She said she became “furious” after her husband defined the phrase as “a part of a woman’s anatomy that is stimulated by foreplay before orgasm.”

6-: From the review of A Dictionary of Euphemisms (London: Hamish Hamilton, 1983), by Judith S. Neaman and Carole G. Silver—review by the English novelist and critic Anthony Burgess (1917-1993), published in The Observer (London, England) of Sunday 14^th August 1983:

There is a large difference between a true euphemism and a word which presses the expressive button (a term given here for clitoris, which is surely too learned a word for the man in the boat and is not improved by being abbreviated to ‘clit’).

7-: From Speaking of Language, by Bill Casselman, published in The Gazette (Montreal, Quebec, Canada) of Saturday 8^th February 1997:

The clitoris, seat of sexual pleasure in women, is “kleitoris” in ancient Greek, meaning literally “the man with the key” or “the gatekeeper,” from “kleis” – key –plus “oris,” Greek agent noun suffix. A playful origin of names for the clitoris is repeated in many languages. Compare a British folk term: “the little man in the boat.”