Edmund was born in 1744. Edmund's father was Edmund Ruffin and his mother was Anne Simmons. His paternal grandparents were Robert Ruffin and Elizabeth Watkins. He had a brother and two sisters, named James, Mary and Elizabeth. He was the youngest of the four children. He died at the age of 63 in 1807.