Of British-English origin, the phrase (to be left) to hold the baby, and its variants, mean (to be left) to deal with an unwelcome responsibility.

This phrase alludes to a stranger’s accidental (as opposed to a parent’s legal) responsibility for an infant—as exemplified by the following, published in The Western Morning News (Plymouth, Devon, England) of Wednesday 10^th June 1874:

A mysterious occurrence took place at Exeter yesterday. A woman, slipping a shilling into the hand of a boy, asked him to hold her baby for a minute, and then hurried away. Finding that she did not return the boy took his little charge to the police-station, where it was found that the shilling had been wrapped in a piece of paper containing an earnest appeal for the child to be temporarily taken care of.

The following from the Daily Mirror (London, England) of Monday 14^th November 1960 reported the same type of incident—with the title punning on the figurative and literal meanings of the phrase:

She is left holding the baby. .
A month-old baby was abandoned yesterday—in the arms of a woman cleaner at King’s Cross station.
A woman who had been feeding the child, a girl, in a waiting room, asked the cleaner to mind the baby while she “made a phone call.”
Then police got an anonymous call that a baby had been abandoned at the station.
They found the cleaner in the waiting room—holding the baby.
The child was taken to a council nursery at Kentish Town.

The texts containing the earliest occurrences of the phrase (to be left) to hold the baby and variants that I have found indicate that this phrase originated in the language of stock markets. These early occurrences are as follows, in chronological order:

1-: From the North British Daily Mail (Glasgow, Lanarkshire, England) of Saturday 28^th September 1872:

SCOTCH IRON.

Looking at the present state of the warrant market, it is extremely difficult to prognosticate the future. We stated in our iron trade report last week that there would be a pressure for delivery of warrant, and how far our views have been correct we allow our readers to judge. From the state of the account the market at the present time is in an anomalous condition. On all sides it is said there is a “rig on,” and somebody must hold the “baby;” but, in the meantime, iron is still going out of store, and the shipments keep up to a fair average price.

2-: From the North British Daily Mail (Glasgow, Lanarkshire, England) of Saturday 16^th November 1872:

SCOTCH IRON.

We have little change to note this week in the general condition of the iron trade, and cannot as yet report any improvement in business, which has been for some weeks very dull. This state of matters is generally expected at this time of the year, and under these circumstances, when there is almost no business doing in the warrant market, and also in most branches of the iron trade, it is difficult to lay before our readers any new feature. The warrant market is, however, assuming rather a more healthy appearance, and no farther great fall in prices is anticipated now that the “bulls” and “bears” have arranged matters. A considerable quantity of iron has been disposed of privately (not in the ring) at a fair price by the parties who were supposed to hold the “baby,” which will considerably reduce their stock of warrants. A great deal of unnecessary harsh language has been used in the Iron Ring against certain parties who have only acted according to the orders given to them by their constituents, stating, also, that by their manipulations trade has been paralysed. The warrant market does not rule the legitimate trade of the country; formerly makers took their cue from the Ring, but during the late rise the price of warrants has been entirely guided by the price of makers’ iron, and with the present high price of coal, and especially the difficulty as to the labour question, it is impossible that makers can reduce the price of pig iron, so as to make it at a profit.

3-: From an article about the rise of rents in Glasgow, published in The Glasgow Herald (Glasgow, Lanarkshire, Scotland) of Friday 26^th February 1875:

There is another reason why rents have risen so rapidly of late, and that is the mania which set in some time ago for speculating in house property. The fever was spread first of all by the success of a few daring and clever operators, some of whom have realised handsome fortunes in a period which seems almost incredibly short. […] The extent to which this practice has prevailed is very considerable, and it led, of course, in many cases to landlords who were not speculators raising their rents likewise. A very pleasant amusement, no doubt, for owners of and traffickers in house property, but the tenants might reasonably say with the frogs in the fable, “What is sport to you is cruelty to us.” There is an end to everything, however, and there is reason to believe that the progress of the property mania is drawing to a close. The shrewder of the operators are getting out quietly, but, in Stock Exchange phrase, some will be left to “hold the baby.”

4-: From a letter, by a person signing themself ‘W. J.’, published in the “Money Market and City Intelligence” section of The Times (London, England) of Friday 20^th December 1878:

“Sir,—Will you allow me to utter a word of warning about the new and plausible Parliamentary scheme for the appointment of an arbitrator in the City of Glasgow Bank’s affairs? […]
[…]
“What would be the probable operation of this Bill? It could only be this—that under its powers the arbitrator could make and enforce calls more rapidly—the object being declared to be rapidity—than the liquidators would, and that creditors who are naturally in despair at the delay in getting their money, and to some of whom such delay may mean ruin, may squeeze it out at the earliest possible date and save themselves from bankruptcy. Let me suppose a case. Let me say that there are a number of bill discounters in London or elsewhere whose readiness to accept its paper without inquiry, in the confidence that some one else would be left ultimately to ‘hold the baby,’ was one main cause of the bank’s ruin, and that they are left holding it, to their great concern.”

5-: From the account of the annual meeting of the electors of the Fifth Ward, Greenock, published in The Greenock Advertiser (Greenock, Renfrewshire, Scotland) of Tuesday 14^th October 1879:

An elector said a question had been suggested to him by Councillor Paton’s answer to a former question. Did the candidates think that another attempt should not be made to secure Cowan’s property at the corner, so that they might have the buildings complete all round?
[…]
Councillor Paton said the Council was quite at one with him when he said that they would like to acquire that property, but the price put upon it, considering the small area, would make it a very expensive addition. Sixteen thousand pounds would be a very heavy burden upon the Municipal Buildings scheme, and, therefore, they thought it their duty to wait. It was just possible that Cowan’s trustees or heirs might consent to sell it for the sum the Council wished. They could do perfectly well without it, only it would improve the appearance of the buildings. (Applause.)
Dean Smith said he could quite agree with the answer given by Councillor Paton. It would be a desirable thing that the Council should be able to get possession of that property, but he would not be a consenting party to give anything like the price asked. Councillor Shankland had referred to the purchase of Purdie’s property. To give them an illustration of the relative value of the two properties, he (Dean Smith) might mention that they had paid for Purdie’s property £28 10s per yard of the ground. That might appear a large sum, but it was not larger than was paid for some other buildings in Hamilton Street by private parties for private purposes. It should be remember that the property was situated in the very centre of the property of the Town Council, and when the Town Council resolved to erect buildings it was absolutely necessary that they should acquire Purdie’s property. They could do without Cowan’s property, but they could not do without Purdie’s. The price put upon Cowan’s property amounted to some £55 per square yard. The price offered was £38 per square yard. There was a limit even to a public conscience, and he was one of those who thought that Cowan’s heirs should be allowed to hold the baby in the meantime. If they were to buy the property and pull it down it would add some £20,000 to the cost of the Municipal Buildings scheme, and he could not see that it would yield anything like a revenue to compensate this outlay.