The phrase don’t give up the, or your, day job is used as a humorous way of recommending someone not to pursue something at which they are unlikely to be successful. The noun day job denotes a person’s regular job and main source of income, as contrasted with a more enjoyable occupation or hobby.

The earliest instance of the phrase that I have found is from New Miltie, He’s So Shy, published in The Billboard (Cincinnati, Ohio) of 16^th June 1951:

New York, June 9.—Tradesters at the Tony Martin-Gene Baylos¹ opening at Bill Miller’s Riviera Thursday (7) saw unveiled at ringside the new Milton Berle². Berle, known from Lindy’s to Lucey’s for his merciless slashing of any working comic who ventures to mix it with him, took several sharp cuts from Baylos without stabbing him back. Said Baylos, “Milton, I Saw Your TV Show. [missing word?] blurred,” and again, “Berle, I caught you on TV. Don’t give up the day job.” And when Tony Martin introduced Berle, calling him “the greatest entertainer I have seen in many, many years,” Milton simply stood up, like a dignified uncle, bowed and sat down. Said one tradester: “Could it be he’s sick?”

¹ American comedian Gene Baylos (1906-2005)
² American comedian Milton Berle (born Mendel Berlinger – 1908-2002)

The phrase expresses a mere common-sense recommendation in Emcee at the Comedy Store: A Comedian Grows in La Jolla Nightclub, by Joan Levine, published in the Los Angeles Times (Los Angeles, California) of Monday 24^th July 1978:

The first advice a fledgling comedian usually gets is, “Don’t give up your day job.” In a world where comics live or die by their punch lines, Larry Himmel³ has no intention of retiring early.
As he explains it, he has been toying with comedy for the past few years. “I see myself as an apprentice in the art of comedy, and will be for years to come.”
As emcee at the Comedy Store in La Jolla, doing an occasional featured stint, Himmel has a built-in advantage that most beginners don’t have. “I’m in a job that allows me to polish my craft, and get paid at the same time. My radio job pays the bills, so I’m really in good shape.
“If I had to make my living telling jokes, I’d be another neurotic comic, and there are tons of them, and not enough work to go around.”
Right now, Himmel has the best of both worlds. As morning man on KGB-AM, he spins records and does sports features with a crazy bunch of characters, including tennis guru Swami Sweatsocks, and ultrasports announcer, Freddy Capistrano.

³ American comedian and journalist Larry Himmel (1946-2014)