Swan Tavern sits on Lot 25 and was first acquired from the Crown Trustees by Charles Hansford on November 24, 1691 but he ‘deserted’ and the lot reverted back to the trustees. Fourteen years later on January 8, 1706, Daniel Taylor purchased the lot for 180 pounds of tobacco. Just one year later he sells to James Sclater. Nine years later it is conveyed to Benjamin Clifton in 1716. It is on August 17, 1719, the first Nelson acquired the property, ‘Scotch Tom’ Nelson and Joseph Walker. Within a year or two, these gentlemen build a structure called the ‘Swan’ which was first opened as a House of Entertainment on March 18, 1722, with Robert Wills as the innkeeper.
The next Nelson, William Nelson, eldest son of ‘Scotch Tom’, buys out Joseph Walker’s interest. In 1761 conveys the lot and building to his son, Thomas Nelson, who becomes Governor and Signer of the Declaration of Independence. The tavern remained in the Nelson family until 1778 when sold for 1,000 pounds to Lawrence Gibbons who then operated for ten years. Next came Lawrence Smith who lived in and operated the tavern from 1788 until 1811. He then sold it one Preacher Scervant Jones.
The Tavern building was a large story-and-a-half frame structure built on substantial brick walls of the basement. The first floor contained four large rooms and included a wide central hallway. The same plan appears on the second floor. One of the rear rooms was used typically for the tap room. This building is similar the Dudley Digges home on Lot 77.
During the Civil War when Union forces occupied Yorktown and used the courthouse as a munitions magazine, there was an accident blowing up this building and subsequent burning of the Swan Tavern. Other building nearby, stable, kitchen, and other outbuildings were also destroyed. Ten years after the Civil War, in 1875, Samuel Brent built the Brent Hotel until 1905. This later building was also destroyed by fire later in the twentieth century. When the National Park Service took over the building prior to 1933, there were no buildings on the lot.
When the National Park Service excavated in 1933 the Swan Tavern foundations were uncovered for the tavern, stable, kitchen, and smokehouse. Using Mathew Brady Civil War images, insurance records, and inventories, all four structures were rebuilt and completed by 1935.