Nate Silver: Dear Media, Stop Freaking Out About Donald Trump’s Polls

[Mr. Morden]

He fails to distinguish betweens true "undecideds" and the fact that some people change their minds during a campaign. The fact that of the 30% that currently support Trump, a fair bunch of them might very well end up supporting someone else does not mean that they are "undecided". They have decided for now, but they might chance their mind.

I'm not sure I agree with that.  When the voting is still months away, I think the polls capture a large number of people who still view their choice of candidate in a highly hypothetical way.  Sure, they give the pollster a name on the phone when they're polled, but it's just a nominal "choice" at the moment, as opposed to a die hard partisan who's pretty solid about their selection.  I think that's the phenomenon that Silver's referring to.

This is why I think it's not exactly clear what early polls are measuring.  "If the election was held today..."  Well, it isn't held today.  And people don't necessarily think about it in the same terms the day before the vote as they do months before the vote.

[Mr. Morden]

And the whole Nate Gospelism thing is that he is better than the media - he can tell when they're going to be wrong and goes against them when they're going to be. This year, all he's done is run with the failed media consensus - he said that Bush and Walker would be top contenders for the nomination, they aren't. He continues to say that Trump will fade shortly, even though all evidence points to Trump making it to at least Super Tuesday - Sure, these are things that Atlas ran with for a time, but we're just commoners - Nate, according to his fans, is supposed to be better than that, and he's not.

As I just said in this thread:

https://uselectionatlas.org/FORUM/index.php?topic=224006.msg4817473#msg4817473

show me where Nate is saying that Trump will fade shortly.  He says he doesn't think that he's going to win the nomination, but explicitly says he's not sure about what stage of the game that's going to happen in, conceding that he could hold on to most/all of his current support into the primaries, and lose the nomination.

As for Bush/Walker...I think you're reading that wrong.  Saying A, B, and C are the strongest contenders for the nomination isn't contradicted when A and B flame out and drop out / fail to win any primaries.  You're not saying that those will be the top three in the end.  You're saying that you think one of them will be the winner, but you don't know which one yet.  The second, third, and fourth place winners in delegates could be candidates X, Y, and Z who you give little chance of winning, and you wouldn't be wrong.  I mean, does anyone dispute that Phil Gramm had a better chance of winning the 1996 GOP nomination than Pat Buchanan?  That's not contradicted by the fact that Buchanan actually won some primaries, while Gramm dropped out after Iowa.  Buchanan never had a real chance at winning the nomination, while Gramm did.

[Mr. Morden]

What falsifiable metric do you propose to measure probabilistic predictions then? I initially proposed looking at whether they would have made money on the betting markets, which initially looked good for Silver & Co. until smilio pointed out that they likely would not have held those shares indefinitely.

I'm suggesting that on this particular topic of Trump's chances at the nomination, it's not falsifiable at all yet because no one has voted yet.  Three or four months from now, it will be.  At that point, he'll be proven right or wrong on Trump.  But since all he's said (as far as I can tell) is that he thinks Trump isn't going to be the nominee, without suggesting either that there's going to be some kind of "collapse" of his existing support or that he knows the moment when another candidate will surpass him, it's premature for people to say he's already been proved wrong.  That's not to say that people shouldn't laugh at his prediction for being ridiculous based on currently available information, if they think it is ridiculous.  But I don't understand people saying that Silver predicted Trump's "collapse was imminent".  Did Silver say that?  Maybe he did, and I missed it.  If I did, where's the quote?

But let's broaden this out from Silver.  Others, like Jonathan Bernstein, are even more "the party decides" fundamentalists than Silver is.  As a group, I think all these people are saying is that the party will ultimately stop candidates it deems unacceptable (such as, presumably, Trump, Carson, Cruz).  That's all.  Not that none of those candidates will ever win any primaries, or get more than 30% of the vote in a state.  Just that they won't win the nomination.  But people here distort that, and suggest that "this person is doomed to lose the nomination" translates into "this person is going to see all of their support erode tomorrow, and will come in 12th place".  When, again, we have many examples of candidates in the past who stood no realistic chance of winning the nomination, yet won multiple primaries.

So sure, maybe they're right and maybe they're wrong.  But saying "the polls have already proved these people wrong" is incorrect.

[Mr. Morden]

Silver's appeal has rested on his basing things on numbers. He's supposed to have done something that allows him to supersede punditry. I very much agree that there is a lot of misunderstanding of statistics out there, and that doesn't help. But if all he's saying is, "I can never be wrong because I never put 100% or 0% odds on anything," then there's not really much value to his brand of analysis. It really comes down to poll averaging, plus a little bit of incorporation of fundamentals.

You can check how good someone is at assigning probabilities by looking at their overall track record.  Do events that they predict to be 80% likely happen 80% of the time?  Do the 90% events happen 90% of the time?, etc.  But you can't pick out a single one of those events and "prove" that the probability assigned was wrong after the fact, because there's no objective check for such a thing.  That doesn't mean that you can't laugh at the assigned probability for being too optimistic or pessimistic, based on the information at hand.  But those assessments are subjective.  That's not a dodge by Silver.  That's just a matter of definition.

I don't have a problem with people laughing at Silver's assessment of the race.  I'm just confused about people making things up about what Silver is predicting (suggesting that "this guy isn't going to win the nomination in the end" means "this guy's support is going to evaporate tomorrow"), and then saying that he's already been "proved wrong" on this because of polls with Trump at 30%.  And again, this isn't just about Silver, but all of the "party decides" people.  They are largely saying that Trump isn't going to win.  Not that he's not going to get a lot of votes.  Let's wait until the votes are counted before declaring them as having been "proved wrong".

[Mr. Morden]

I'm not at all saying it's a dodge. I understand the math, and I understand how probability works. I'm just wondering whether there's value in it to justify the reputation he got as some sort of wunderkind.

Yeah, sorry, my comment in that regard was more a response to other comments in the thread than to you.

[Mr. Morden]

Yeah. The data is there if someone wants to do it. I'm not sure if the senate models were updated daily, but I think it was weekly or maybe a few times a week. Anyway, with several elections now and all those Senate races, there's enough to do it in a statistically significant manner. It would be a lot of work though.

Someone here did it for just one snapshot of time for one year's Senate races, and found 538's probabilities to be OK.  But it was a very small sample size taken from just one election year.  There's also the fact that the probabilities might be excellent one month before election day but terrible six months before election day, or vice versa.  The accuracy could go up and down depending on how close to an election you are.

One problem I have with the probabilities they have in some past cycles is that there are numerous cases where Silver's model gives a heavy favorite something like a 99% chance of winning early on, but then the race tightens later, and those odds drop substantially.  If you're seeing 1% events turn into 30% events with relative frequency, that would tell me that maybe you're being too quick to hand out 99/1 from the beginning.  That shouldn't happen that often.  If the early polls are only so predictive, then you really shouldn't be seeing too many races rated 99% that early.