Polarization and partisanship matter much more than incumbency.
This hasn't always been true, though. Why did they manifest to such an extent this year?
Well, it was very true in 2016. We've just become very divided and uninterested in other points of view, and more and more races have become nationalized. You saw it in the House too, with a lot of Republican incumbents in Democratic-leaning districts doing worse than Trump, even though Trump did badly in their districts. It was certainly quite extreme this year, but it's not exactly sudden. We're been getting progressively more polarized and partisan for years now.
