Robert [de Toeni], called de Stafford; held by 1086 nearly 70 manors in Staffs, more than 25 in Warwicks, more than 20 in Lincs, 10 in Oxon, one in Worcs and one in Northants; built what later became known as Belvoir Castle; allegedly married Avice de Clare, and died probably 1088. [Burke's Peerage]
--------------------------------------------
Burke's Peerage has Robert de Stafford and Robert de Toeni of Belvoir as the same person. However nothing is certain about that. Many people feel that they were two different men.
-------------------------------------------
The first that assumed the surname of Stafford was Robert de Stafford, who possessed, at the time of the General Survey, lordships in Suffolk, Gloucestershire, Lincolnshire, Warwickshire, and Staffordshire, in all 131, and Dugdale surmises that the assumption of the surname of Stafford arose from his being governor of Stafford Castle, which had been erected by the Conqueror; for his name originally was de Toenei, and he is said to have been a younger son of Roger de Toenei, standard bearer of Normandy. of this Robert de Stafford, who lived till Henry I's time, nothing further is known than his founding an Augustine priory at Stone, in Staffordshire, upon the spot where Enysan de Waltone, one of the companions of the Conqueror, had killed two nuns and a priest. He m. Avice de Clare, and was s. by his son, Nicholas de Stafford. [Sir Bernard Burke, Dormant, Abeyant, Forfeited, and Extinct Peerages, Burke's Peerage, Ltd., London, 1883, p. 498, Stafford, Barons Stafford, Earls of Stafford, &c.]