Despite several claims to the contrary, we have no evidence of Thomas Jefferson's personal interest in the game of marbles or ownership of a collection of marbles. The first known appearance of this information is in 1972; it was repeated in Fred Ferretti's 1973 book The Great American Marble Book and in a subsequent article by Ferretti on marbles for the Encyclopedia Americana.
There are a number of letters where Jefferson refers to "marbles," (e.g. "I have not a scrip of a pen on the subject of the marbles you mention as arrived for me at Philada., neither invoice, nor advice nor letter of any sort.") and although these are references to mineralogical specimens, it is possible that these references may have contributed to the confusion on this topic.