The colloquial phrase to stick out, or to stand out, like a sore thumb means to be very obviously different from the surrounding people or things; it is especially used of someone or something ugly or unwelcome.
—Cf. also like a pork chop in a synagogue.

The first instance of to stick out, or to stand out, like a sore thumb recorded in the Oxford English Dictionary (2^nd edition, 1989) dates from 1936, but the phrase is in reality much older, since the earliest occurrence that I have found is from The Summit County Beacon (Akron, Ohio) of Thursday 16^th July 1868:

The “Pendleton Club.”

Never was there a more disconsolate looking set of fellows, since the surrender of Lee and Johnson, than the little handful of copperhead wire pullers, yclept the Pendleton Club of this city, on the nomination of Horatio Seymour, on Thursday last. The blow was stunning in the extreme, creating ‘nary’ a shout (except from the Republicans) nor even a smile of approbation, but on the contrary producing the self-same style of countenance that used to come over them on the receipt of the news of a Union victory during the late “onpleasantness.” By and by, however, the dirty rag, which had been “sticking out like a sore thumb” for several weeks, with the name of George H. Pendleton emblazoned thereon; was quietly hauled into the club room window, and the name of the new, but evidently unwelcome, patron saint of the unwashed but no longer “unterrified,” Horatio Seymour, shoved out. The nominations came down upon the democracy here, as all over the country, like an ice-cold wet blanket, producing a political ague chill that not even daily double doses of the customary party tonic can break between this and the second Tuesday of November next.

The second-earliest instance that I have found is from a Letter from Joe, dated 25^th May 1869 and entitled A Tale of Two Cities, published in the Owyhee Semi-weekly Tidal Wave (Silver City, Idaho) of Tuesday 29^th June 1869:

There is great jealousy on the part of Cincinnati toward Chicago—it sticks out like a sore thumb; the loose morals of Chicago—natural to a place of such rapid growth—is a source of endless comment and jest in the press of Cincinnati.

I have also discovered a phrase that is not recorded in the Oxford English Dictionary: to be on hand like a sore thumb, meaning to be fully available.

The earliest occurrence that I have found is from the following paragraph (difficult to understand, because of the switch from past to future) published in the Columbia Democrat (Bloomsburg, Pennsylvania) of Saturday 28^th July 1849:

Europe, Asia and Africa, were on hand, on last Friday, July 27^th at least in the representative capacity of the Lions, Tigers, Rhinocereses, Elephantes, snakes and monkies. Of course you will all be there. There will be a great many things done, beside many that will not be done. You must be on hand like a sore thumb.

This paragraph apparently referred to the following advertisement published in the same issue of the Columbia Democrat:

RAYMOND & Co’s

MENAGERIE,
Containing the Rarest Collection of
WILD BEASTS, BIRDS, AND REPTILES,
Ever before Exhibited in the United States
Will exhibited at Bloomsburg, On Friday July 27^th, 1849.

The second-earliest instance of to be on hand like a sore thumb is from a toast made during the supper taken at the City Hotel by the printers of Manchester, “an annual affair in honor of the Birthday of Franklin”—published in the Manchester Daily Mirror (Manchester, New Hampshire) of Monday 19^th January 1857:

The Editors, Publishers and Printers of Manchester: On the Anniversary of Franklin’s Birthday, may they ever be like a sore thumb, always on hand.