Quotation: “In matters of style, swim with the current; in matters of principle, stand like a rock.”
Sources consulted (searching “stand like a rock” and “swim with the current”):
Earliest known appearance in print: 1891 (as the second variation) 
Earliest known appearance in print, attributed to Jefferson: 1973 (as the first variation) 
Comments: It is clear that the quotation came into use at least as early as the 19th century, although when it was used, it was not attributed to a particular author and was often referred to as an “old adage.” It is not clear where the phrase originated from, but there is no proof that Jefferson ever uttered these words. It appears that the phrase became connected to Jefferson around 1973, and from then on, it is almost always attributed to him when quoted - usually in the context of homemaking or education.
- Elizabeth Huff, June 8, 2011
931 Thomas Jefferson Parkway
Charlottesville, VA 22902