How Nate Silver Missed Donald Trump

[Alcon]

Silver made two fundamental errors with regards to Trump.

1) He basically chose to ignore polls and go with his gut feeling, thereby repeating the mistake that he has often himself accused other political analysts of making and neglecting the qualities which made him a good number cruncher in the first place. Silvers forté was always in crunching numbers. As a political analyst he is no better than all the others who go by their gut and then interpret the facts to allign with their gut. This is the stuff that Dick Morris is made of.

2) He NEVER understood the appeal of Trump. The appeal of Trump is not that he is a celebrity. It is not that he is anti-establishment. It is not that he is a novelty. It is not that he gets 90% of the media attention. All of those facts contribute to his popularity, but the heart of his appeal is his alpha male persona. True alpha male politicians can be incredibly popular with the general public, as shown by the likes of Berlusconi in Italy, Putin in Russia or indeed, Hitler in Germany.

I agree with all of this.  Silver's Trump downplaying has always been weird to me because there are so many variables involved, and Silver has always been a guy who emphasizes the statistical value of uncertainty.  The guy's model still gives Carson and Rubio a 9% chance of winning in Iowa.

I don't object to the existence of Silver's polls-plus model, although it seems to me -- as someone who barely understands it -- that he's weighting the "plus" part too heavily, especially this close to the election.  It's totally fine to find a hypothesis supported by past data, and try to implement it in a model.  It certainly has a good shot of having more predictive power than polls do, especially months out.  However, you have to incorporate the uncertainty of such a multi-variable approach, plus the added risk of overfitting.

For a guy who's so conscious of uncertainty, I'm surprised he's fallen for such conventional traps of the statistically illiterate.  That said, I still appreciate him because, even when I think his methodology is messed up, at least I know he has a methodology.  I still appreciate the poll-based prediction he publishes and the modeling he does.  Even if I think he's fallen for the traps pundits do, he still contributes way more of value to the discourse than most pundits.  He's too much maligned here.

[Alcon]

Silver made two fundamental errors with regards to Trump.

1) He basically chose to ignore polls and go with his gut feeling, thereby repeating the mistake that he has often himself accused other political analysts of making and neglecting the qualities which made him a good number cruncher in the first place. Silvers forté was always in crunching numbers. As a political analyst he is no better than all the others who go by their gut and then interpret the facts to allign with their gut. This is the stuff that Dick Morris is made of.

2) He NEVER understood the appeal of Trump. The appeal of Trump is not that he is a celebrity. It is not that he is anti-establishment. It is not that he is a novelty. It is not that he gets 90% of the media attention. All of those facts contribute to his popularity, but the heart of his appeal is his alpha male persona. True alpha male politicians can be incredibly popular with the general public, as shown by the likes of Berlusconi in Italy, Putin in Russia or indeed, Hitler in Germany.

I agree with all of this.  Silver's Trump downplaying has always been weird to me because there are so many variables involved, and Silver has always been a guy who emphasizes the statistical value of uncertainty.  The guy's model still gives Carson and Rubio a 9% chance of winning in Iowa.

I don't object to the existence of Silver's polls-plus model, although it seems to me -- as someone who barely understands it -- that he's weighting the "plus" part too heavily, especially this close to the election.  It's totally fine to find a hypothesis supported by past data, and try to implement it in a model.  It certainly has a good shot of having more predictive power than polls do, especially months out.  However, you have to incorporate the uncertainty of such a multi-variable approach, plus the added risk of overfitting.

For a guy who's so conscious of uncertainty, I'm surprised he's fallen for such conventional traps of the statistically illiterate.  That said, I still appreciate him because, even when I think his methodology is messed up, at least I know he has a methodology.  I still appreciate the poll-based prediction he publishes and the modeling he does.  Even if I think he's fallen for the traps pundits do, he still contributes way more of value to the discourse than most pundits.  He's too much maligned here.

Someone contributing more valuable discourse than the pundits isn't saying very much though.

To each their own, but I think conversations like this are much more useful than 99% of punditing.  I'd rather be in a room of five Nate Silver types of varying methodologies and arguments than someone whose entire methodology and argument is gut instinct.  Silver is at least someone who cares about having a methodology, and is willing to explore the contingencies if his approach is wrong.  Politics has plenty of people whose every word comes straight from the spleen, and these people are the worst.

Basically, the criticisms I've seen of him lately seem to based on the idea that Silver is failing to sufficiently emphasize the positives that made him valuable (focus on methodology, empiricism, contingency-based thinking).  Basically, Silver has turned into diluted Silver.  I might agree -- but I'll still take diluted Silver over the gut-rotting punditry that prevails out there (which is often presented much more arrogantly).  He's getting way too much crap.

[Alcon]

Heh, can't argue with that

[Alcon]

Basically, the criticisms I've seen of him lately seem to based on the idea that Silver is failing to sufficiently emphasize the positives that made him valuable (focus on methodology, empiricism, contingency-based thinking).

It's not simply that. It's also that he's using dubious statistical methods in the pursuit of a political agenda, namely Silver's personal antipathy toward Trump and bias toward Rubio that he's expressed on multiple occasions, when he had previously been more or less "objective" in his application. It's not that he's diluted, it's that he's violating the "Do No Evil" principle in applied statistics.

I was being unfair -- I should have said "most of the criticisms," not "the criticisms."  You're presenting a totally legitimate criticism, and hell, I agree with it.  I also disapprove of his "editorial hand" this year.  However, at the end of the day, at least he tells me what his editorial hand is and why (even if he's done a fairly weak job of applying scrutiny to it), which means he's still a valuable input.  I'd rather deal with someone who operates in hypotheses and models than people who don't understand how that thinking works.

The reaction to Silver this year reminds me of how gleeful people tend to be when they find out, like, that a priest shacked up with a woman.  The response sometimes seems less about how bad the offense was, and more about people laying scorn on someone who had the audacity to be put on a pedestal by others.

[Alcon]

Remember folks, Trump is just going to be the flavor of the month, Jindal still has a chance at winning Iowa, and Jim Webb will be the anti-Hillary. 538 can't be wrong when they talk out of their ass with no evidence since they managed to average polls a couple of days before a general election and not be totally wrong.

Thank you for completely wasting your intelligence in this thread.

[Alcon]

@Gustaf: Exactly. Although the actual mechanics of endorsement points are pretty rough (notice how each is worth 1, 5, or 10 points -- obviously not an advanced algorithm), there is an historical association between endorsements and increasing support which suggests they can be a leading indicator, not just two things that generally go hand-in-hand.  You could reasonably argue that this linkage only applies to conventional candidates, and conventional candidates rarely lose primaries, but...that's Silver's point.  And it's well-documented here.  This is historically how things work.

The truth is that any model of such a complicated phenomenon is going to fail to accurately gauge someone whose electoral appeal is incredibly anomalous.  And that's fine, because good models shouldn't necessarily predict anomalies, especially if it models something super-complex like a primary.  Sometimes adding variables meant to predict, say, a Donald Trump, will throw the model off in the 90%+ of other cases would have functioned quite well.  That's unreasonable, and it's usually especially unreasonable to include it in the model if it's never happened in the data set.

I think Silver's 'mistake' this year was in arguing that the lack of previous anomalies meant it was unreasonable to assume Donald Trump was an anomaly.  Think about that for a minute, though: how often do claims that a candidate is "the exception" to a longstanding pattern turn out to be true?  Almost never.  Even an "anomalous" candidate like Obama fits into the model fine.

So, I think Silver's mistake was more in being dismissive (understandably) and not being willing enough to discuss the possibility of Trump being a candidate that presents unknown unknowns, probably because he's allergic to people invoking that kind of argument to justify magical thinking all the time, and almost always ending up wrong.  Even if I think he dug his heels in for a little too long (and even if I think his model should be producing more uncertainty on the endorsement factor when it hasn't apparently kicked in this close to the first vote), I understand where Silver is coming from.

@Figs: Here ya go.

[Alcon]

@Figs: Here ya go.

That's interesting and goes part of the way toward what I was wondering, but it also doesn't necessarily give me an indication of whether endorsements as a variable is separable from polling support. He sketches a very, very rough idea of how endorsements might translate to votes, but I still don't have a good handle on exactly how much that effect might be captured by, or duplicating, other data.

I'm struck in particular by the Democratic charts for 2004. Kerry's endorsements spiked after he started succeeding in primaries, before which point they were lagging Dean and even Gephardt.

You're right that he doesn't show it's an independent variable that's discretely influencing outcomes -- but if it's a leading indicator, it seems like it's reasonable to model.  If you look at 2004, it strikes me that several candidates had comparable establishment support and the establishment started coalescing around Kerry, and then he continued to grow momentum.  It's true that elite endorsements weren't as much as a leading indicator there, since the establishment appears to have stayed out early on, but once they started to get behind Kerry, his momentum did continue and accelerate.  And he was never really disfavored the way Trump (and Cruz, for that matter) is.  Kerry isn't much of a knock on the model, IMO.

That scenario is a lot different than having continued, escalating momentum for someone with consistent zero establishment support (Trump).  That has literally never happened before in the data set, even in a more limited way.  Consider also that some people managed to gain apparent momentum, were then ignored by the establishment, and then flopped.  Santorum in 2012 (after his Iowa win) is one of several good counterpoints.

Looking at all of this, I really think it's hard to convict Silver of much more than misdemeanor analytic stubbornness.

[Alcon]

But isn't this (similarly to Obama's increasing number of endorsements in '08) more a case of "the establishment" backing Kerry since he had gained momentum and appearing increasingly likely to win, rather than the endorsements causing him to win the nomination?  The same seemingly applies to 1988 as well (in which both Gephardt and Gore had more endorsement points than Dukakis shortly before Iowa).  

Yes, except the point of the model is that candidates who gained momentum and then received institutional endorsements tend to continue the momentum...those who don't, do not.  Hence my reference to Santorum, among others.  You're effectively arguing it's exclusively a trailing indicator, but the evidence doesn't really gel with that.

***

1-He never did this in the past. He may have tracked endorsements in 2008 but only because it mattered for superdelegates. I don't recall him ever doing so in 2012. And in 2008 Silver was actually pretty skeptical of the media's whenever they did one of those "Hey Politician X from state that's coming up to vote just endorsed Hillary/Obama DOES THIS CHANGE EVERYTHING?" stories.

The first part is irrelevant.  Debate the merits of the model, not the motivation.  The second part is a strawman of how the model works.

2-Just look at it. You not only have Jeb! in first and Trump with zero, but Christie, Kasich and HUCKABEE are even beating Cruz. This has about zero relevance to the actual campaign.

You're accusing him of adopting an unreasonable model based on the current campaign.  No one is contesting the model has done a poor predictive job this year.  We're discussing whether it was reasonable going into the campaign.  No one in this thread is arguing that Silver shouldn't have been louder and more transparent about the "unknown unknowns" relating to the model, especially as it became increasingly obvious those were a big problem for it this year.

3-He's assigning objective numerical values to endorsements like that can be done. It reminds me of how teenage posters like to say things"If *candidate* picks *Governor/Senator* as their running mate, then they will gain X% in that home state and Y% in neighboring state." Yes VP picks matter but not like that. And similarly it's kind of silly to say that the Governor of Idaho or Wyoming's endorsement is worth 10x as much as some influential Tea Party House Republican's, or Charlie Baker and Larry Hogan's too for that matter. I'll note this is the only reason why Christie is so high, he has two Governors. Kasich and Huckabee too benefit from Gubernatorial endorsements. The Governors for Christie are...oh yeah actually Larry Hogan! And Paul LePage. Huckabee has his own Governor of Arkansas. Kasich has the Governor of Alabama. who care.

Of course, but how could you practically quantify that without introducing massive problems of subjectivity?  The idea behind a model is to build the most reasonable predictive tool.  Pointing out the flaws of the model doesn't mean it should be trashed, unless you can find a superior methodology.  

Frankly, I think it's pretty clear that Silver weighted the model in a way where (even up until near the last minute) it still had a lot of influence on his polls-plus predictions.  I think the polls-plus prediction model seems to fail to incorporate enough uncertainty about its own hypotheses, especially as its hypotheses failed to predict things this entire year.  However, you're going way beyond that and arguing the model itself was consistently so fatally flawed that it was never a reasonable model.  I disagree.  You haven't really proven that case.

Pointing out that any individual portion of the model is rough is a little like pointing out that a Fermi estimate is made up of a bunch of approximate estimates.  Yes, of course it is...that's what a Fermi estimate is, and likewise, most models of complicated multi-variable phenomena are pretty rough.  And of course more precise estimates would be better.  The point is that imprecise estimates are often the best we have -- and Silver had historical data to support assuming that to be the case here.  It doesn't quite justify his stubbornness in discussing this year, but that's better methodology than 95%+ of 'political analysts' bother with.