Benjamin Cardozo was openly agnostic. Obviously agnostic and atheist are not the same thing, but as I've said before, irreligiosity was not always the massive political liability in America that it became between ~1960 and 2010.
The man who replaced Cardozo on the Court was Felix Frankfurter, who was also ethnically Jewish like Cardozo, but was non-practicing. Frankfurter has been quoted as saying, upon the death of either Chief Justice Stone or Chief Justice Vinson, that said death "was the only evidence that I have ever seen that there is a God."
