Nate Silver: Dear Media, Stop Freaking Out About Donald Trump’s Polls

[Slander and/or Libel]

Silver is desperately grasping to retain his reputation. He's tried to position himself as something different than just a pundit, but he went out on the same limb as everyone else early in the process, saying that Trump had to collapse. He can't admit that he was just shooting from the hip, spouting his opinion, because that's off-brand. But he also can't sit back and not address how wrong he seems to be, so he has to come out with this, telling everybody to believe him and not their lying eyes.

[Slander and/or Libel]

What value is Silver providing? If he's not more right than other pundits, what's the point of him?

[Slander and/or Libel]

Silver's appeal has rested on his basing things on numbers. He's supposed to have done something that allows him to supersede punditry. I very much agree that there is a lot of misunderstanding of statistics out there, and that doesn't help. But if all he's saying is, "I can never be wrong because I never put 100% or 0% odds on anything," then there's not really much value to his brand of analysis. It really comes down to poll averaging, plus a little bit of incorporation of fundamentals.

I've personally wondered if somebody could go back and mark the point at which his predictions shift to correct (defined in this case as greater than 50% chance of what will ultimately occur occurring) and never shift back. If he "called" all 50 states correctly by this metric for, say, the last month running up to the election, that would be impressive. But if his model just follows the shifts in the polls, and winds up being an average of them, then what's the point?

[Slander and/or Libel]

Silver's appeal has rested on his basing things on numbers. He's supposed to have done something that allows him to supersede punditry. I very much agree that there is a lot of misunderstanding of statistics out there, and that doesn't help. But if all he's saying is, "I can never be wrong because I never put 100% or 0% odds on anything," then there's not really much value to his brand of analysis. It really comes down to poll averaging, plus a little bit of incorporation of fundamentals.

You can check how good someone is at assigning probabilities by looking at their overall track record.  Do events that they predict to be 80% likely happen 80% of the time?  Do the 90% events happen 90% of the time?, etc.  But you can't pick out a single one of those events and "prove" that the probability assigned was wrong after the fact, because there's no objective check for such a thing.  That doesn't mean that you can't laugh at the assigned probability for being too optimistic or pessimistic, based on the information at hand.  Those assessments are subjective, by definition.  That's not a dodge by Silver.  That's just a matter of definition.

I don't have a problem with people laughing at Silver's assessment of the race.  I'm just confused about people making things up about what Silver is predicting (suggesting that "this guy isn't going to win the nomination in the end" means "this guy's support is going to evaporate tomorrow"), and then saying that he's already been "proved wrong" on this because of polls with Trump at 30%.  And again, this isn't just about Silver, but all of the "party decides" people.  They are largely saying that Trump isn't going to win.  Not that he's not going to get a lot of votes.  Let's wait until the votes are counted before declaring them as having been "proved wrong".

I'm not at all saying it's a dodge. I understand the math, and I understand how probability works. I'm just wondering whether there's value in it to justify the reputation he got as some sort of wunderkind.

[Slander and/or Libel]

Fair enough. But, at least on the issue of Bush and Walker, it seems like a convenient cop-out regarding their respective flame-outs to say "I only gave them 1 in 4 odds each."

And what would you like him to say? Yes I totally blew it on Walker?

Suppose Walker won and someone calls him out on it, since he predicted there was a 75% chance that wouldn't happen. What should he say then?

If he predicts Walker has a 25% chance of winning, that means he thinks Walker won't win. So why should we rag on him when that comes to pass?

But by that logic you could say he predicted that someone other than everyone would win, right? He thought it was more likely that anybody else would win than that any individual candidate would win.

I understand the math, and I understand what he's doing. I just think that argument is a little disingenuous, is all.

[Slander and/or Libel]

Can that even be done in a meaningful manner, though? That is, is there really a big enough sample size? He puts "if the election were held today" percentages on races for months in advance of elections, but those percentages wouldn't be meaningful in this kind of analysis, because the election was not held on that day. The only percentages that'd be meaningful would be the ones right before the election, which are by that point bare poll averages.

Unless I'm missing something, of course.

[Slander and/or Libel]

Can that even be done in a meaningful manner, though? That is, is there really a big enough sample size? He puts "if the election were held today" percentages on races for months in advance of elections, but those percentages wouldn't be meaningful in this kind of analysis, because the election was not held on that day. The only percentages that'd be meaningful would be the ones right before the election, which are by that point bare poll averages.

Unless I'm missing something, of course.

That is not correct, as far as I can tell. To quote Silver himself, regarding his predictive model for the 2014 Senate elections: "The FiveThirtyEight model is explicitly meant to be a forecast of how the election will turn out on Nov. 4, 2014 — rather than an estimate of what would happen in an election held today."


http://fivethirtyeight.com/features/how-the-fivethirtyeight-senate-forecast-model-works/

Ah, thank you for that clarification. I'd been misremembering.

So theoretically, then, Silver has provided us with REAMS of data to confirm his models, right? He updated them once a day in the runup to the elections, right?

[Slander and/or Libel]

I kind of disagree. In primaries there is less and less reliable polling so that approach is a lot less valid. What he did was to look at historical numbers with predictive value - like endorsement points and such. And based on that he was willing to distrust the polls.

Obviously, this was wrong, in hindsight. But I don't think it was as dumb or ludicrous as people make out now.

Not being sure means putting big error bars on predictions, not claiming that one outcome is super likely.