‌Joseph and Jane were married in a religious ceremony on May 1st, 1804.  They had five sons and four daughters, named William, Samuel, Andrew Cavendish, Jacob, Joseph, Abigail, Nancy, Jane and Elizabeth.