In 1824 Henry Lee leased the land from Trinity College and built this house for himself. He and his brother were brickmakers and lime merchants at Loampit Hill from 1793. The family remained in the house until 1858.

James Allen, the managing director of P&O, lived here from 1861 until his death in 1874.

In 1883 the name was changed to The Clock House and much later to Canada House. The house might have been used to billet Canadian troups during World War I and between the wars it fell into decay, being used by a glass bottle merchant and then a garage.

During World War II it was used by builders and deteriorated further. In 1947 it was demolished and replaced by Hether Grove Estate.