According to some family, Hugh was a mean man who would beat his wife Alice. Dorothy Wilson, one of Hugh and Alice's daughters came to live with Alexander J. "Bill" Warren and Estelle Riley Warren to "get away from that family." Hugh was dark complected and "looked like an Indian."