James didn't have his first 3 children baptised until 1817. This may have been because he wasnt married. Therefore a marriage to Mary or Nancy at Bury (or Accrington N of Ramsbottom) in 1816 is a good match. On 29 Apr 1816, James Mather, a Dyer married Mary Stott at Bury
7 May 1827, James and Mary (both 40) and children John (14), Elizabeth (10), Hannah (10), Rachel (8), Martha (6), Sarah (5), James (3) and Mary (1), all from Ramsbottom sailed to NY, USA.
In 1860, James and Mary E (81 & 71) were in Dunbar, Fayete, Pennsylvania but this may be another copule, born in the USA.