The AAPOR report on 2016 “polling failure”

[Mr. Morden]

Anyone taken a look at this new report yet?  There are some summaries here:

http://www.politico.com/story/2017/05/04/2016-election-pollsters-react-237975

https://www.washingtonpost.com/politics/were-the-polls-way-off-in-2016-a-new-report-offers-a-mixed-answer/2017/05/04/a80440a0-30d6-11e7-9534-00e4656c22aa_story.html

Sounds like they’re saying that the polling problems of last year came about not because of any kind of “Shy Trump voter” effect, in the sense that people were lying to pollsters about their voting intentions, but because of sample selection problems.  (Which is exactly what I was saying months ago.)

Specifically, the issue was that national pollsters were more likely to weight by education level than state pollsters, which was a problem for polling in the Rust Belt, where there was an especially high undercount of voters without a college degree:

You must be logged in to read this quote.

[Pericles]

Their conclusion on Comey is wrong. They fail to note that while Clinton was trending downward slightly before the Comey letter, her numbers crashed after the letter and she lost over half her lead in less than a week. On October 28, her lead was 4.6%, just 5 days later it was 1.7%. On October 23 her lead was 5.6%, so Clinton's fall accelerated after the letter. Her lead of 3.2% on Election Day(RCP average) was 1.4% smaller than before the Comey letter. While some of that support was recovered, her lead was still significantly smaller on Election Day than before the letter was released. It's also possible that the slight national polling error was caused by Comey exonerating her(which he would never have needed to do if he hadn't sent out the letter in the first place!) and bringing 'Clinton emails FBI' back into the headlines and the minds of voters right before they went to the ballot. I'd say Comey cost Clinton Michigan, Wisconsin, Pennsylvania and Florida. Nate Silver agrees Comey cost her the election, and she was quoting him when she said that if the election was on October 27 she would be President. And given the closeness of the election, even if Comey only shaved 1% of her lead and had a small effect, that would have been enough to flip the result and I'd say that the effect was higher.

[mencken]

Isn't demographic weighting itself a pretty shoddy practice? It is essentially increasing the effective margin of error of the poll to avoid ugly-looking crosstabs.

You must be logged in to read this quote.

This is especially rich, considering Trafalgar had the best Michigan poll (beating PPP, Fox, and implicitly every national pollster who thought Michigan was uncompetitive), and the best Pennsylvania poll (beating Quinnipiac, CNN, CBS, and NYT). Ohio was a much tighter race in the polls conducted by CBS, Quinnipiac, Suffolk, CNN, and NBC (the latter two of whom did not even bother to poll after mid-October) than in such luminaries as Remington, Trafalgar, and Emerson. The winners in North Carolina were once again Trafalgar and Remington, beating out Quinnipiac, NYT, CBS, and NBC.

[Mr. Morden]

Isn't demographic weighting itself a pretty shoddy practice? It is essentially increasing the effective margin of error of the poll to avoid ugly-looking crosstabs.

If you don't do any demographic weighting then you just get complete junk, since some demographic groups are markedly more likely to respond to polls than others.

[Southern Senator North Carolina Yankee]

So in some ways, it was a similar class skew that occurred in 1936 with Literary Digest, just not quite as substantial an error.

[Associate Justice PiT]

So in some ways, it was a similar class skew that occurred in 1936 with Literary Digest, just not quite as substantial an error.

     Limiting the size of the error had something to do with the fault being much more regionalized here than it was in the Literary Digest poll. If you look at states like Wisconsin, the polling was way off.

[I Won - Get Over It]

And I don't think that this is a new issue.

I guess the "missing" uneducated voters have not voted homogeneously before why one couldn't see the results of this issue. The thing is that in 2016 they did.

Actually, I think that there were the same problem in BREXIT.

[Mr. Morden]

And I don't think that this is a new issue.

I guess the "missing" uneducated voters have not voted homogeneously before why one couldn't see the results of this issue. The thing is that in 2016 they did.

Yes, that is the issue.  There wasn't that much of an education gap between the parties as recently as the 2012 presidential election.  So if your polling screwed up the educational weighting or didn't do it at all, then it wasn't likely to impact the topline number.  But since 2016 saw a big education gap between the parties, not doing any educational weighting now creates a big bias.