In 1888 the renowned surgeon Sir Jonathan Hutchinson founded the museum in the grounds of his home at Inval in Haslemere. Determined to offer educational opportunities to local people he allowed the public to enjoy his life's collection of objects of interest in the field of botany, geology and social history. The huge popularity of the Museum led to the launch of an innovative series of lectures and activities which still continue today. In 1895 the collections moved to Museum Hill and in 1926 the Museum arrived at its present site in the High Street where it has become a popular landmark for the town.

The oldest part of the current building dates back to the sixteenth century and although it was altered in the middle of the nineteenth century the facade is little changed from its original appearance in Georgian times. The extensive grounds and garden contain several unusual trees and shrubs including a gingko tree. A wonderful view towards Lythe Hill can be gained from the historic garden with its curved Ha Ha and visitors can take a picturesque walk through the woods past the Gazebo and pond.