As already alluded to up above, McReynolds is basically the Scrooge of the Court's history: nobody on the Court even came to his funeral, whereas the funeral of his Black messenger, Harry Parker, was attended by like 6 justices, including the Chief Justice. Needless to say, pretty much
everybody on the Court hated his f**king guts, & not least because the contempt was not only very much mutual but at first impression too: he was such an anti-Semite that after Brandeis was confirmed to be the first Jewish justice, McReynolds would leave the conference room whenever it was Brandeis' turn to speak & even refused to allow the Court's picture to be taken one year since the seniority protocols dictated that he'd have to sit next to Brandeis. He proceeded to refuse to speak to Brandeis for the latter's first 3 years on the bench, & never once to Cardozo.
Frankfurter was also a mess of drama: not only was he one of the many justices who derided Douglas as sloppy & too motivated by his political ambitions, having written that "Douglas' crookery has a depth beyond plumbing. One has had to live with him for years to appreciate his shamelessness," but he himself was so hated that his departure actually precipitated a notable increase in civility on the Court. As long as he was on the Court, though, there were
always rivalries on it, with Frankfurter being widely disliked, particularly by McReynolds & Black, because he talked down to his colleagues on the Court. Douglas also got in one "f**k you" at Frankfurter, infamously refusing to sign Frankfurter's retirement card, like how McReynolds had refused to sign Brandeis', just without the anti-Semitism.
Also, I don't think anybody who's ever worked with Burger could be said to be a fan of his. By all accounts, he was a jerk, plus such an incompetent Chief: he couldn't manage a conference to save his life & who'd
blatantly change sides on a case just to control the opinion assignment. Stewart probably would've killed him himself had he stayed on the Court any longer.
Also, said justices were widely disliked by most of those with whom they served, even if it didn't extend to pure "hatred." Since Burger left, though, there haven't been any publicly revealed personal feuds between the justices, with seemingly all of the justices being very emphatic that everybody on the Court works to gets along with one another in spite of any ideological differences, which is frankly remarkable, given the lack of civility in the Taft/Hughes/Stone/Vinson/Warren eras.
Frankfurter really didn't like the more liberal on the court, particularly Douglas and Black, but also Warren.
Black also didn’t get along with Jackson (and vice-versa)
Getting to avoid Black was apparently one of the primary reasons that Jackson accepted the offer to prosecute at Nuremberg, & both of them said that they would immediately retire if the other were to ever be promoted to Chief.