Samuel's father was James Mather and his mother was Alice Lord. His paternal grandparents were Samuel Mather and Betty Lord. He had a brother and a sister, named James and Sarah. He was the youngest of the three children. He had two half-brothers named Samuel and James.