Adsmore was originally constructed as the Greek Revival-style residence of dry goods merchant John Higgins in 1854. James B. Hewitt owned the residence from about 1870 until 1900. At that time, it was sold to John Parker Smith, of the prominent Smith-Garrett family, who employed Brinton B. Davis to enlarge it in his noted Colonial Revival style. The house passed to Smith’s daughter, Mayme (Smith) Garrett, on his death. Garrett’s daughter, Katherine Garrett, inherited Adsmore and made it her home until her death in September 1984. She bequeathed the Adsmore estate to the trustees of the George Coon Public Library. Her will stipulated that all of its elaborate furnishings to be restored and that the house be maintained as a public museum. Along with the residence, her endowment for the operation of a museum is said to be over $1,000,000.