The Bad Boy

When James and Dolley Madison moved to the White House officially on March 4, 1809, they were accompanied by her son Payne Todd, child of her first marriage. Payne had turned 17 only a few days before and had lived with his mother and adoptive father in Washington already for nearly eight years, ever in the shadows of the prominent and highly social Madisons. As secretary of state, the highest-ranking cabinet member, and close personal friend of President Thomas Jefferson, James Madison was very much a presence in the new capital. Whether Payne was close in the family circle or rather pushed out by its other commitments is not known for certain, but the latter can be suspected as obvious treatment for a child in such a situation.

If Payne had a lucky star, it was at his birth. His father John Todd was a young and rising Philadelphia lawyer, married to Dolley Payne. They were active Quakers, at home in a large community of Quakers, some of whom, like her family, had left the South to find lives more compatible to their beliefs in Philadelphia. Notwithstanding Quaker rules of simplicity, the Todds lived in some affluence, occupying in the city a handsome three-story corner house of brick. A deadly yellow fever epidemic swept Philadelphia in 1793, sending the family in flight. John Todd daringly returned on business and was struck down. Dolley Todd’s infant son died the same day. Dolley Todd survived. Playful and social, she and her sister Lucy received too many men callers to please the Quaker Meeting and were cast out. At her home she met James Madison after only a year of widowhood. They escaped to Virginia where they were married. Madison gave her a ring of uncut diamonds; she wore it thereafter, with miniatures of both husbands. Two years later Madison adopted little Payne Todd. It was more a declaration of adoption, one supposes, for the child’s name was not changed to Madison.

For the school months of the family’s first years in Washington, Payne Todd boarded at Saint Mary’s Seminary, a Roman Catholic institution in Baltimore. When a resident of the White House, he commuted daily to school downriver. The Alexandria Academy, which still stands in Old Town, had been endowed by George Washington and held sway as one of the best schools in the Washington area. While Payne Todd had a taste for fine art, he seems to have had no time for academic study.

Payne Todd was a good-looking young man, his miniature portrait reminiscent of his late father’s. His mother’s charm was not lost in him, or her potato nose that artists usually improved in her portraits. And he was bright enough. Jefferson invited him to dine. Payne Todd surely felt comfortable to an extent in the company of the people who swarmed to the White House. They were eager to please him. But for a presidential child inexperienced in public life, the White House can be a perilous place. Both good and bad sought him out, and Payne seems to have been drawn to the latter.

President Madison, worried about young Todd’s conduct, dispatched him on a State Department mission in Russia and another in Europe, thinking he would improve in the broader world. Instead of the anticipated good results, the experiences could not have turned out worse. Todd tasted a succession of evils and relished them all. He returned to America with habits. These were to follow him all his life, bringing destruction to his mother. She professed to love him above all else, and was to come to his need. Who can imagine how deep this affection ran? His character had been shaped by the forces of his growing up. Perhaps Dolley Madison felt some guilt.

By the time the Madisons retired to their farm, Montpelier, in Orange County, Virginia, where the president had been raised, Todd’s activities had become a curse. He lost money gambling, womanized, and while drunk was so handy with a gun that he was jailed several times for disturbing the peace. Madison came up with the bailbonds. At last, at another effort to straighten him out, Madison put him in charge of his farms. He made him his secretary, again in good hopes. Todd had no aptitude for farming or management, or for being an amanuensis. Whether through fear of mistakes or simple inability, he failed at everything. John Quincy Adams wrote of him that his “tendency towards dissipation” seemed “irresistible” and that he had a “moral incapacity for industry and application.” There may have been to Payne Todd more than met the eye. If so, none saw it.

Yet Payne Todd worried about the future: what would happen to him after James Madison was dead? The force of his life was, in this instance, not entirely negative but included his mother’s well-being. His diaries do show that he tried. He dealt in sheep, hogs, and various grain crops. In a context of a falling national economy, his efforts seem those of a desperate but also unaware individual, for the economic times would have tested the most astute overseer of a fortune in farms. Madison’s estate had suffered early from the Panic of 1819. The collapse of the banks in the 1830s during Jackson’s administration exemplified the decline of federal management into gross politics, a painful reality for Madison the Founding Father. The former president’s death in 1836 at Montpelier threw the estate into the hands of Dolley Madison, who, for all practical purposes, turned most of the management of the farms and other endeavors over to her son.

The widow’s ownership of her husband’s estate was not free and clear. Part of the indebtedness was hers and Madison’s own free hand loaning to friends, part was their extravagant entertaining, part was the general picture of the economy. Payne Todd spoke of taking their assets and joining the flood from Virginia to new investments in the agricultural Flush Times of the Deep South. We do not know the extent to which this idea was pursued. More indebtedness sank Montpelier into an inevitable loss, and the farm was sold by Dolley Madison in 1844. Payne held back some furnishings, perhaps to sell in a more favorable market. Those materials, statuary, and paintings, Dolley Madison determined to sell by a raffle.

During her hard times in the later 1830s the former first lady visited Washington with some frequency, a legendary character from another time. She returned to Washington permanently at the age of 76 for the last five years of her life, to occupy a house she owned on Lafayette Square across from the White House. With little effort she created a new presence for herself in political society. Payne Todd wove in and out of her final days, always affectionate toward his mother while laying his gambling debts before her to pay. His friends and associates were of his kind. Urgency to pay gambling debts carried the threat of pistol and knife. He was weak and probably afraid.

It had been both Dolley Madison’s and her late husband’s desire to sell James Madison’s papers to raise much-needed funds to live on. Madison had kept notes of the Constitutional Convention; although doing so had been prohibited by the rules of the convention, everyone eventually knew about it and the value of the notes to the interpretation of the Constitution was very great and well known. Money, however, proved hard for Dolley Madison to raise for the papers. She kept the several thousand pages of notes, letters, and the like in a trunk. One night in Washington her house caught fire. Her one concern was saving the trunk of papers, which a fellow occupant of the house shouldered downstairs to the street.

The papers were sold to Congress in two batches, the first thanks largely to Henry Clay and Daniel Webster, although many others, such as James Buchanan, spoke in favor of the purchase. The purchase bill was passed in December 1837. Notwithstanding numerous important documents Payne withdrew on the sly and stashed for his own benefit, Dolley Madison received a very generous $25,000. This was an amount equal to the president’s yearly salary. She wished to take $5,000 to pay debts and live on interest from the balance. Despite a second sale of additional papers in 1848, her easy street was not to be paved. Debts of her own combined with the apparently ever-present gambling debts of her son took most of the money very fast. A small percentage, however, survived at her death. For all the fruits of depravation Payne Todd loaded upon his mother, she never ceased expressing her affection for him. Friends’ objections were strong but were kept among themselves.

At last she died in her house on Lafayette Square. Her funeral, held at Saint John’s Church on the square, was all but a state affair, and after being in a temporary tomb her remains were moved to Montpelier to lie beside James Madison’s unmarked grave. Payne Todd was left with a large portion of the estate, including her house in Washington and parcels of land left at Montpelier. The money went fast. Payne Todd was at last thrown on the world. Alcoholic and fast spending his inheritance, he reached the age of 60 and died a month after. In his will he freed his slaves, perhaps his most noble act. He is buried in Congressional Cemetery, Washington.

Payne Todd’s story is not usual to the children of presidents after the White House. Most children of presidential households have gone on to live relatively quiet lives. While there is no obligation to do so, some even have led illustrious careers. Two have become presidents themselves; writing, business, social work, and public careers have been enjoyed by White House children. They have been able to manage themselves and their conduct. Payne Todd was not. With the likely exception of neglect at home, for his errant conduct was carried out always away from home, his shortcomings cannot be blamed on the position of his parents but on his own weaknesses. Of all the children of the White House, Payne Todd is remembered most of all as the bad boy.