Robert was born in 1705. Robert's father was John Carter and his mother was Grace White. His paternal grandfather was Robert Carter; his maternal grandfather was John White. He had three brothers and three sisters, named Nathaniel, John, William, Mary, Martha and Ann. He was the third oldest of the seven children.