Republicans didn’t prioritize this race early enough (allowing Peters to define himself/James before they realized that MI was in fact more winnable than CO) + not the best R messaging + decline of split-ticket voting in general. Also, unlike Peters, James was largely invisible on the campaign trail.
James ran for the other Senate Seat in 2018, so I have a feeling he was already "defined" in the eyes of the electorate, so this probably wasn't the problem. The reason he lost was probably that he didn't distinguish himself enough from Trump in the state that's soured on him the most of the big 3. And yet like most GOP senate candidates, he still outran Trump.