Born in Charlottesville, Fountain was a descendant of Wormley Hughes and Ursula Granger his grandparents were amid the residence slaves owned by Jefferson at Monticello. Jefferson allowed two of Sally's children to go away Monticello without the need of formal manumission when they came of age 5 other slaves, together with the two remaining sons of Sally, ended up freed by his will upon his dying.