Charles was born on October 15th, 1791 and his baptism took place on October 20th, 1791 in Docking St Mary. Charles' father was Thomas Callaby and his mother was Susan Hull. His maternal grandparents were William Hull and Ann. He had three brothers and two sisters, named Thomas, Matthew, William, Jane and Amy. He was the youngest of the six children.