In cricket, from the resemblance between the figure 0 and a duck’s egg, the term duck’s egg denotes the zero (i.e. 0) placed against a batsman’s name in the scoring sheet when he fails to score. This was explained in Very Hard Cash (1863), a novel by the English novelist and playwright Charles Reade (1814-84):

(edition: Harpers & Brothers, New York, 1864)
“You and I, at Lord’s the other day, played in first-rate form, left shoulder well up, and achieved—with neatness, precision, dexterity, and dispatch—the British duck’s-egg.”
“Misericorde! What is that?” inquired Mrs. Dodd.
“Why, a round 0,” said the other Oxonian, coming to his friend’s aid.
“And what is that, pray?”
Alfred told her “the round 0,” which had yielded to “the duck’s-egg,” and was becoming obsolete, meant the cipher set by the scorer against a player’s name, who is out without making a run.
“I see,” sighed Mrs. Dodd: “It penetrates to your very sports and games. And why British?”
“Oh, ‘British’ is redundant: thrown in by the universities.”
“But what does it mean?”
“It means nothing. That is the beauty of it. British is inserted in imitation of our idols, the Greeks; they adored redundancy.”

The term duck’s egg has been shortened to duck, as explained in British Sports and Pastimes (London, 1868), reprinted from Saint Paul’s Magazine, edited by Anthony Trollope:

It is amusing to see how the characteristics of different men show themselves in such an arena, and how opportunity is given for the exhibition of that which in ordinary life stands concealed. For instance, it has often been observed that a very conceited man, who seems to be shamelessly bumptious, is really the most nervous of creatures. At cricket this is detected to a certainty. […] If you wish to see a real funker*, look at him when the dreaded moment arrives […]. You see by the twitch of the hand, the glove rapidly raised to the face, and replaced on the bat-handle, the jerk of the elbow, and perhaps the uneasy lifting of the foot, that his fear of a “duck”—as by a pardonable contraction from duck-egg, a nought is called in cricket-play—outweighs all other earthly considerations.

(* funker: a coward; the noun funk, in the sense of cowering fear, was first mentioned as Oxford slang in 1743.)

The phrase to break one’s duck(’s egg), from the idea of breaking the 0, means to score one’s first run in an innings, thus avoiding a ‘duck’. In Guide to the Cricket Ground (London and Cambridge, 1867), George H. Selkirk wrote:

Duck’s Egg—When a batsman makes 0 in an innings. If he makes one run he has ‘broken his duck’s egg;’ and if he makes 0 in each innings he is said to have made a ‘pair of spectacles.’

This phrase has come to mean, generally, to achieve a particular feat for the first time. The earliest instance of this extended use that I have found is from the South London Press of 7^th July 1894; the proceedings of the Rotherhithe Vestry (= meeting of parishioners for the conduct of parochial business) contain the following:

Mr. Payne, L.C.C. [= London County Council], […] had already taken action with respect to obtaining more chairs for the Southwark Park band, and as to these he confessed he should like to see some little charge made, so that they might be retained for those who attended the performances with the object of listening to the band.
Mr. Stuart was glad to find Mr. Payne had broken his “duck’s egg” at the Council. (Laughter.) He hoped the hon. gentleman would support the vestry in its effort to obtain a road through the park from the main road to the Southwark Park entrance. If Mr. Payne persevered in that matter, he ought to be successful.

Similarly, to score a duck is used in the general sense of to fail to score, as in this article from the Evening Telegraph & Star (Sheffield, Yorkshire) of 26^th September 1888:

If “Britons dearly love a lord,” there are some of Her Majesty’s subjects who decidedly object to titled fame. Among that noble minority is Alfred Wilmott, of St. Albans. So bitter is Alfred Wilmott’s antipathy to the aristocracy that he actually disdains to be called by a nickname which faintly suggests that he has some connection with the peerage. Addicted to the fascinating game of billiards, the bold democrat was engaged the other day in a match with a friend at a local hotel. Suddenly there drew nigh unto him a scoffer in the person of one Alfred Webdale, who straightway hailed Mr. Wilmott by the opprobrious epithet of “The Baron.” Instead of being gratified by that aristocratic address, Alfred Wilmott’s passions were stirred to their lowest depths. His finest plebeian instincts were outraged by finding himself dubbed “The Baron,” and his soul at once sprang to arms. So did his hand, for he slipped it into his pocket and pulled out a revolver, with which began to take pot shots at the offending Webdale. Fortunately, Mr. Wilmott’s aim with the pistol was not equal to that with the cue, and by Webdale ducking his head beneath the table, the infuriated marksman scored a “duck.” That was not the end of the affair, however, for Alfred Wilmott has since had to appear before the magistrates.