“Scotch Tom” Nelson built his son, Thomas, a brick house about 1725, according to the current marker on the site. The land was given to Secretary Nelson in 1744 and assumed the house was built sometime after this date. This substantial mansion served as the residence of Lord Cornwallis and other British officers leading up to the siege of Yorktown. 
Thomas Nelson, known as Secretary Thomas Nelson, uncle of Thomas Nelson, Jr., Governor of Virginia and Signer of the Declaration of Independence, became the colony’s secretary of state (hence the designation “Secretary Nelson.” Secretary Thomas Nelson, getting along in age, was in residence when the British rode into town.
The house was bombarded in October 1781 during the siege of Yorktown. Secretary Nelson was in the house when the siege started. During the attack, his butler was killed serving dinner. It was then Secretary Nelson, October 10th, he left the house under a truce flag and escorted by his three sons through the American lines, who were officers under General Washington.
The home was substantial with a room two-story dormered structure, with a double-hipped roof and four large chimneys. The foundation measured 56 by 49 feet. There were four out buildings as well as gardens. The two-hipped roof structure was designed to capture water and deliver into a cistern. All that remains today is the outline of the home, destroyed during the siege. A color sketch by Benjamin Latrobe around 1796 shows the house pot marked in every direction with cannon-shot. Well into the nineteenth century were references to the ruin still remaining but in deteriorated condition. 
With the Association of Preservation of Virginia Antiquities (APVA) branch established in Yorktown in 1921 there was interest for identifying and saving the Secretary Thomas Nelson home site. It was not until 1924 that the APVA Directress was directed to write a letter of thanks to the owner of the land, Mr. John F. Braun, who uncovered the foundation and placed a sign on the site at his own expense. Apparently the Virginia State Highway Department in 1928 was planning to build through the foundations of the home. The APVA appealed to the APVA General Association, to the Governor, and to appropriate Congressmen to preserve the site. Commissioner H. G. Shirley of the Highway Department was sympathetic and suggested the APVA Branch try to acquire the site. This proved successful and the Branch received the site as a donation in 1928 to include five feet outside the foundation. A cover coping of concrete placed on the top of the old foundation would help safeguard it from weather as well as those collecting souvenirs.
On May 6, 1933, a tablet was dedicated to the Secretary Thomas Nelson home. The marker reads: Foundations of the Home of
President of the Council and
Secretary of State
of the Colony of Virginia
Erected in 1725
First headquarters of Lord Cornwallis
It was Destroyed During
The Siege of Yorktown
Preserved and Marked by the Yorktown Branch
Association for the Preservation of Virginia Antiquities, 1930
Today the site can easily be missed as one drives from the Colonial Parkway and approaches the Yorktown Victory Monument. A small pull off provides an opportunity to read an information sign. Take time to ponder the moment when artillery was fired into the home as America won its independence.
 Colonial National Historic Park (CNHP), Yorktown’s Main Street Historic Resource Study, Part one, Yorktown’s Main Street (From Secretary Nelson’s to the Windmill). Chapter III – The Secretary Nelson Mansion, March 1974, http://www.nps.gov/history/history/online_books/colo/yorktowns_main_street/chap1-3.htm
 Ibid, CNHP, Part one.
 Page Genealogy of Virginia, by Richard Channing Moore Page, Pg 143.
 Ibid, CNHP. Part one.
 Colonial National Historic Park (CNHP), Yorktown’s Main Street Historic Resource Study, Appendix C – The APVA Engineers the Acquisition and Marking of the Secretary Nelson Home Site., March 1974, http://www.nps.gov/history/history/online_books/colo/yorktowns_main_street/appc.htm