Systematic State-Level Polling Bias

[wbrocks67]

In examining things, I feel like there needs to be a bigger emphasis on specifics. When you talk about the polling, most people need to realize that it any polling "misses" were mostly from being inundated with garbage GOP stuff.

The *nonpartisan polling* was pretty fantastic overall this year on average. However, there doesn't seem to be as much of a serious understanding among pundits and 538/Cook/etc that they really let the GOP stuff that was clearly trash have way too much of an impact whether on their punditry or their models. 538's model having Oz+1 by the end was possibly one of the most egregious examples when it was based on just about 90% GOP polling and only 1 or 2 nonpartisan polls by the end. There doesn't seem to be as much of a postmortem as to why they let their model be so affected by these polls and whether or not to even include some of these firms in the future (i.e. Patriot Polling, with no background whatsoever)

I would also say it's interesting that some of the usual culprits of polling misfires recently (PA, MI, WI, etc.) were either on point in nonpartisan polling for the most part (PA) or like you said, Dems were actually underrated (MI, WI) which was interesting.

Colorado also now seems like a blue state that also seems to overperform polling. Dems overperformed in both 2020 and 2022 there.

[wbrocks67]

wbrocks67 is right - it does appear that the traditional, methodologically competent/transparent polling outlets did a good job correcting for error in the 2022 cycle (internally, on the other hand, it appears we overcorrected - but I'm not quite ready to talk more about that with confidence yet). But these outlets also polled way less frequently in 2022 than in previous cycles, so it's possible they just had fewer chances to be wrong. Hard to tell for sure.

Regardless, polling bias is becoming more fluid and I think a model to predict it, while informative and helpful, will be difficult to get right by any metric.

The good news is that polling bias does not affect directional results within a poll and therefore keeps it incredibly useful and valuable for campaigns (and keeps me in a job).

Please keep us updated on your findings internally! Do you think it was a systemic thing across the board (i.e. national Dems, Dem groups etc. also overcorrected a little too much?)

There's also another piece to this - in that there seems to be serious polling biases among certain groups like young voters and minorities. Quite a lot of the polling issues across the board this year came from terrible sampling among these groups. Some pollsters did get it right, so it IS possible, even with small sample sizes. But I think a lot of pollsters need to figure out how to reach these groups. If you get a survey back and it has the Republican getting 25-30% of the black vote in a swing state, or tied among 18-29 year olds with the Democrat, something is seriously off. Though, I think many of the GOP pollsters were just making these #s up. The biggest nonpartisan offender I think was Emerson.