Herbert was born in 1858. Herbert's father was George Hayhurst and his mother was Sarah Mather. His maternal grandparents were Robert Mather and Grace Wood. He had a brother and two sisters, named Frederick, Mary and Annie. He was the second oldest of the four known children.