2020 Polling Errors by State

[S019]

So, over the past several weeks, I have become increasingly annoyed by takes both here and on Discord that polling is worthless and that you should add 5 points to every Republican in polls and whatnot, so with some free time on my hands, I decided to look back at the polls taken just before the election (i.e. the last 10 days) in each of the 50 states:

For the purposes of this project, I chose to use a larger polling bias to mean that the polling average was further away from the actual result, also in terms of the map coloration, if it is colored for Democrats, it means that Democrats did better in polls than they did on Election Day, and vice versa, if it is colored for Republicans


So, here is what I found:



Key for the shadings:

10+: 100%
8-10: 90%
6-8: 80%
4-6: 70%
2-4: 60%
1-2: 50%
0.5-1: 40%
0-0.5: 30%

Couple of interesting patterns, first that polling tended to be worst in low population safe states, unsurprisingly there was a lack of polls in these states leading up to the election. Also there was no polling at all for NE-03 so that district has not been colored in as Republican victory there was assured, basically no matter what.

So, now if we limit this to states that had at least 5 polls in the pre-election period defined above, we get this:


What can be immediately noticed is the dark maroon in the Midwest suggesting a strong polling bias towards the Democrats, which is absolutely true, however it must be remembered that polling, like any form of statistical analysis, has a margin of error, a common one used is 3%, in the case of a tied poll of 45-45, the true value can lie anywhere from 48-42 in one direction or 48-42 in the other, which in reality looks like a 6 point miss. While this may explain the more modest misses in Michigan and Florida, it does not do much to explain the strong misses in Ohio, Wisconsin, and Iowa. Elsewhere, in the heavily polled swing states polling fared quite well, managing to basically nail the exact margins in Nevada and Georgia.

Overall, it seems logical to conclude that outside of three specific Midwestern states, polling fared about as well as you'd expect in the 2020 elections, polling can only provide an estimate of the outcome, and it seems that the polling average was close enough to the actual result in most of the heavily polled states. On the note of Ohio and Iowa, these states also suffered from very significant polling biases in 2018, so it may indeed not be as reliable of a tool for these specific states, but it should still be treated as an accurate prediction of voting intention in most states.

[kwabbit]

Not accusing you of this, but most people have a terrible understanding of what a poll's MOE means. When a poll is 45-45 with a MOE of 3 points, that doesn't mean 48-42 is as likely to occur as 45-45. It means that there is a 95% confidence that the true result, which the poll attempts to measure, is between 48-42 and 42-48.

The true result is distributed normally, with the poll's finding as the estimated mean and the MOE/2 (1.96 to be exact) as the standard deviation of the true result.

So no, 47-43 and 45-45 are not the same result just because they fall within each other's MOE. There's a small chance they are finding the same result plus some variance, but a greater chance they are finding different results.

Additionally, with many polls in the state, the MOE of the polling average decreases. With 10 polls, the MOE will be around 1 point. With the flurry of polls that are released near election day, a polling average will have a MOE that is negligible. That means that the aggregator will be very, very close to representing the actual picture of the race an infinite amount of polling would reveal.

Any difference between the polling average and the actual result of the election are thus not the result of error stemming from variance with random sampling, but are from flaws in polling methodology. These aren't errors that can be solved simply by adding more polls or by increasing the number of respondents in the poll. The pollsters just aren't polling the state correctly.

This is why the errors were a lot bigger in the Midwest and smaller in the South, despite there being plenty of polls in each. Pollsters have no idea how to capture the sentiment of non-College Whites, but since there are less of them in Georgia or Texas than in Iowa, Ohio, Wisconsin, the polling average was more accurate.