George Hughes was related to two important enslaved families at Monticello, the Hemings family through his father and the Granger family through his mother, Ursula Granger Hughes (1787–after 1847). After Jefferson’s death in 1826, Hughes, his mother, and his siblings remained in slavery at Edgehill, the plantation of Jefferson's grandson, Thomas Jefferson Randolph; his father was given his freedom unofficially.
After Emancipation in 1865, George Hughes was a farm manager at Edgehill, while his wife, Sarah Jane, was cook at the Edgehill School for Girls. Hughes was a deacon of the Union Run Baptist church pastored by his brother Rev. Robert Hughes. In 1870 George Hughes and his friend Lewis Hern, grandson of Monticello slaves David and Isabel Hern, made a successful bid for one hundred acres of Albemarle County farmland. Hughes and Hern (Hearns) descendants still live on the property today.