538 Model Megathread

[Slander and/or Libel]

The main difference between polls only and polls plus, by my understanding, is that polls plus bakes in an expected reversion to the mean: the "fundamentals" of the race (i.e. the economy is doing decently but this would be a 3rd consecutive term for the democrats) suggest this should be a close race, and polls plus gives some weight to that fact.

Of course, I would think that the real fundamental of this race is that Donald Trump is the Republican nominee, and that the natural "mean" in such a race is a 10-point loss for Trump.  Of course, such thinking wasn't really helpful in the primary.

I think that's part of the problem with some of these prognostications, but I don't quite know how to quantify it. Whatever the polls say right now, my thinking is based on 2012, and these couple of facts:

Romney lost to Obama by 4 points.
With demographic change, in 2016, that would have been 5 points.
Romney won 27% of Hispanics, Trump recently polled at 19%.
Romney won 56% of college educated whites, Trump recently polled at 42%.

Those latter numbers are subject to change, but they get at what I'm thinking here. I think 2012 is a decent baseline to start working from, or at least as good as anything else at this point. So Trump's got to make up a 5 point deficit before he even worries about starting to make up the deficits that are unique to him compared to Romney. I honestly have a difficult time seeing it happen.

[Slander and/or Libel]

Polls plus is a proven failure. I'll take the bettors consensus over that at this point, and I find them to be really irrational and uninformed/out of touch generally

No. Different polls plus than the primary model.

Note: polls plus is identical to the model that called 49/50 in 2008 and 50/50 in 2012.

Polls plus becomes more or less identical to polls-only by election day. The fact that "polls plus" got it right on election day is attributable only to polling.

Not to mention Silver had a 50.3% chance of Obama winning Florida in 2012. That's flipping a coin that it would take thousands of flips to determine is slightly weighted one way. In what sense was that a "call"?

[Slander and/or Libel]

It's funny when people pretend like 538's models represent reality, rather than an attempt to estimate reality, based on a basket of historical data, and assumptions about how to apply it.

[Slander and/or Libel]

BLUE ARIZONA ON NOWCAST!!
BLUE ARIZONA ON NOWCAST!!
BLUE ARIZONA ON NOWCAST!!
BLUE ARIZONA ON NOWCAST!!
BLUE ARIZONA ON NOWCAST!!
BLUE ARIZONA ON NOWCAST!!

OMGOMGOMGOMGOMGOMGOMGG

Ok, I'm gonna calm my bits now.

Clinton is now winning Arizona in the now-cast.

Arizona is blue on the nowcast.

It's as though people don't read the rest of the thread.

[Slander and/or Libel]

Decided to make a map of all the states that have never dipped below an 80% chance of the leading candidate winning for more than a week:

Clinton — 176
Trump — 146
tossup — 216

This is the most farcical definition of "tossup" I've seen in a long, long time.

[Slander and/or Libel]

Your post literally says the word tossup.

[Slander and/or Libel]

Silver essentially called Florida a coin flip in 2012. He had it at 50.3% for Obama. It is not in any meaningful sense a "call" that was "correct", nor would it have been wrong had Romney won Florida.

[Slander and/or Libel]

While we're comparing the two, I'll admit that I'm a tad shocked that Wang's model hasn't back off at all. Still 97 random/99 Bayesian, as it was a week ago. I thought it might revert a bit, but not so far.

Wang's model has insanely small confidence levels.  Didn't he have the Dems as like 90% favorites to maintain the Senate for most of 2014, but flipped it a few days before the elections?

As I recall, his model was way optimistic for the Democrats right through the elections. He's gone into a good bit of detail about where the failures were in 2014 and what he's done to address them.

[Slander and/or Libel]

It doesn't make any sense to only rate the accuracy of the various models on the basis of the predictions they're making on election day itself.  The models also make predictions a week before election day, a month before election day, four months before election day, etc.  The model that's most accurate on election day itself might not be the most accurate a month out.  And I think it's more interesting to look at accuracy from a month out or a week out.  After all, if you've already reached election day, then how accurate your model is is no longer a very interesting question, as you're going to know the results in a few hours anyway.

This is one thing I've been wondering a lot about. I think somebody needs to develop a methodology for measuring time-based performance of these models. I've thought that maybe one metric would be showing how long before the election the model decided a correct outcome was more likely than not and didn't shift back. That has the disadvantage of treating anything over 50% as a "call," but it would be illustrative, I think. If one model wound up saying Florida was 50.3% for Obama, but half a day earlier had said it was 50.2% for Romney, then that's not a very valuable call. But if it had said for two weeks that it was likelier than not to go for Obama, we could have some more confidence in it as a predictive model.

[Slander and/or Libel]

Seeing a Trump fan call him Daddy is the creepiest thing I've seen all week, so thanks for that.

[Slander and/or Libel]

Nowcast has NV FL NC OH IA all leaning Trump.  Some by the narrowest of margins of course.

Input garbage, output garbage. Only Nate could have his model claim Trump's leading NV and then release an article about how Clinton is likely to exceed her already positive NV polls.

The primary broke Nate.

The fact that he's resorting to quoting political futures markets to back his models is telling. Also the fact that he's basing his uncertainty on polling data going back to 1972 betrays that he really still wants the lesson of the primary to have been that nobody can know a thing, rather than that he just got it wrong.

[Slander and/or Libel]

Anyone know precisely how the model handles correlation between states? I know he talked about it a bit, but it still feels hazy. When he's noting a trend in a state, is it a trend from previous polling, or from his adjustment to the polling, or from his adjustment due to correlation with polling from other states or nationally? It's good to note that states are correlated, but I think it quite likely that by putting his thumb on that scale so hard, he's double counting some things, or canceling out others.

[Slander and/or Libel]

IIRC, the old polls from the same pollster always get less weight at that moment when a newer poll is added to the database.

It might be a part of explanation.

I don't think you need that to explain it.  As I said, shifts or around ~0.5% in the win probability are going to happen even when you don't add any new polls at all, simply because of statistical noise.  And that's an average.  Sometimes the "phantom shifts," that cannot be explained by any particular poll, are larger than that.

OK. 10000 seems like a lot for 0.5% shift to me. And in absolutely most cases, the changes seemed reasonable.

Remember that 0.5% means a change in 50 runs out of 10,000. It's not actually a big jump at all.

[Slander and/or Libel]

10k simulations only takes a couple of minutes.

[Slander and/or Libel]

Listening to Harry Enten talk, it sounds like he low key doesn't agree with Nate': model in several respects.

[Slander and/or Libel]

She regained Florida in the polls-only model, but I have to say, the huge D-house-effects they've slapped on some of these higher quality polls are absolutely laughable:

It's not like it makes the hugest absolute difference, but it's really quite ridiculous. The act of aggregating the polls is enough to smooth out the differences. Instead Nate needs to say what the polls REALLY said, and then say that some polls are more equal than others, etc. Those two things combined brings the average margin in the 10 polls above from Clinton +0.7 to Trump +0.6.

I'm not really convinced at all that these methods have a ton of rigor, and especially not in combination with each other. There is a huge chance that his estimates are double counting things that are already showing up in the polls on their own.

[Slander and/or Libel]

I feel like Silver came closer to the others not because he detected something about the electorate that they didn't, but because his assumption of huge error bars made everything go closer to 50-50. He was more right, but only trivially so.

[Slander and/or Libel]

I feel like Silver came closer to the others not because he detected something about the electorate that they didn't, but because his assumption of huge error bars made everything go closer to 50-50. He was more right, but only trivially so.

But he had much smaller error bars in 2012, which also looked like a much closer election going in. So yeah, he detected there was a lot more uncertainty this time.

But was there more uncertainty this time (lower precision), or were the estimates just centered at the wrong place (lower accuracy)? These differences matter. Putting everything down to greater uncertainty implies that things were still centered at the right place.

[Slander and/or Libel]

I feel like Silver came closer to the others not because he detected something about the electorate that they didn't, but because his assumption of huge error bars made everything go closer to 50-50. He was more right, but only trivially so.

But he had much smaller error bars in 2012, which also looked like a much closer election going in. So yeah, he detected there was a lot more uncertainty this time.

But was there more uncertainty this time (lower precision), or were the estimates just centered at the wrong place (lower accuracy)? These differences matter. Putting everything down to greater uncertainty implies that things were still centered at the right place.

There was definitely more uncertainty. Polling numbers bounced back and forth a lot more than in 2012 when they were almost perfectly stable all through, and there was a much higher number of undecideds and 3rd party voters. Those are the two main cause of polling errors.

Other aggregators disagreed about the level of uncertainty. And anyway, the uncertainty that mattered in Nate's model was the amount that he let today's results drift to get to Election Day. If you're centered off target, decreasing precision is definitely going to make you hit the target more often, but not because your model is better calibrated at hitting that target. Only because making things more scattershot means that if you're aiming at the wrong place, a bigger error means you'll more often be accidentally right.

It may be the case that this is the proper way to set things, and no election prediction should really be above 70%. But it's worth understanding that Nate's model was more right about Trump not because it saw something specific, but because he calibrated it to be way less specific.