Frankfurter really didn't like the more liberal on the court, particularly Douglas and Black, but also Warren.
I believe MarkD has pointed out that in Frankfurter's memoirs he calls Douglas one of only two genuinely evil men he had ever met, the other being some private citizen or other whom Frankfurter grants the dignity of namelessness.
In addition to Frankfurter's hostility to Douglas and McReynolds's universal unpopularity with his colleagues, it's becoming increasingly obvious that Kagan can't stand Kavanaugh. Stephen Johnson Field in the late nineteenth century is also said to have been a widely disliked man.
Stephen Johnson Field is a name I've never heard before. TIL.
Why would you say he was so unpopular?