538 Model Megathread

[Mallow]

This is why it was pointless to start this before the conventions. Why bother showing these massive swings we all knew were coming?

The polls-plus version has been remarkably steady.

[Mallow]

I'm not sure what polls are responsible, but 538's model has Trump still surging.

[Mallow]

I'm not sure what polls are responsible, but 538's model has Trump still surging.

That's what I'm scratching my head over. I get that he includes all the funky tracking polls, but you'd think more traditional polls would get weighted higher

Even the tracking polls don't show Trump expanding his lead anymore, though. I wish there was a place on the site where you could see a list of the most recent additions to the model forecast.

[Mallow]

I'm not sure what polls are responsible, but 538's model has Trump still surging.

In this case it's ipsos national poll and also these FL polls not weighted too much to move the needle there for Clinton.

The Ipsos looks like the main culprit. However, it was conducted mostly in the midst of Deplorables/Pneumoniagate, so perhaps the improvement since then will show up more in the coming days.

[Mallow]

I'm not sure what polls are responsible, but 538's model has Trump still surging.

That's what I'm scratching my head over. I get that he includes all the funky tracking polls, but you'd think more traditional polls would get weighted higher

Even the tracking polls don't show Trump expanding his lead anymore, though. I wish there was a place on the site where you could see a list of the most recent additions to the model forecast.

It's right here: http://projects.fivethirtyeight.com/2016-election-forecast/updates/

Thank you. Wish I had seen this earlier.

[Mallow]

I'm not sure what polls are responsible, but 538's model has Trump still surging.

In this case it's ipsos national poll and also these FL polls not weighted too much to move the needle there for Clinton.

The Ipsos looks like the main culprit. However, it was conducted mostly in the midst of Deplorables/Pneumoniagate, so perhaps the improvement since then will show up more in the coming days.

Nope, it's 15-19 september. So starts thursday. It's during birther issue if anything

For some reason I thought deplorables/pneumonia was more recent... obviously it's not, as Sept. 11 was the start of the pneumonia issue. I stand corrected.

[Mallow]

If you look at every other probability-based model (NYT Upshot, Sam Wang, Daily Kos), none of them have swung as wildly as Silver's. They have mostly stayed within a 70-90% Clinton range, which makes sense because the fundamentals of the race haven't really changed that much. But a good model that doesn't swing wildly from day to day doesn't bring in the page views. Nate just wants clicks. He's not interested in accurately forecasting anything.

People who think 538 is intentionally being misleading are annoying. Their methodology was largely determined long ago, they haven't changed anything substantial as far as I'm aware (though based on some recent commentary on here, maybe I missed something?). Their model is showing what their model is showing, for better or worse. Complain about their methodology all you want, but to claim intentional bias for page clicks is dumb.

As for a possible explanation for the recent Trump swing, I think this is a possible culprit: the 538 model relies pretty substantially on the "trend" line. The Ipsos poll was added in three sections, the first being the 9/13-9/17 results, and the last being the 9/15-9/19 results. The three polls are Clinton +2, Trump +1, and Trump +2, so it looks to the model like a strong short-term trend from a highly-rated pollster.

[Mallow]

Another huge difference is that Silver's model assumes that the states move together and do not act totally independent of each other. If Trump gains in North Carolina, the model assumes that he also makes some gains in Virginia, unless/until there's polling evidence to contradict that. Not the case with other models.

Yes, which is something that always bothered me. Like... does a bad IA poll impact MO, IL, MN?

Really? That's one of the model's main benefits to me. Assuming the states move independently makes no sense.

[Mallow]

Another huge difference is that Silver's model assumes that the states move together and do not act totally independent of each other. If Trump gains in North Carolina, the model assumes that he also makes some gains in Virginia, unless/until there's polling evidence to contradict that. Not the case with other models.

Yes, which is something that always bothered me. Like... does a bad IA poll impact MO, IL, MN?

Really? That's one of the model's main benefits to me. Assuming the states move independently makes no sense.

I agree to a point, although assigning the proper amount of correlation between the states will be tricky; for example, MS and AL are going to be more alike in their movements than MD and WV, to pick another pair of neighbors.  But this also has the potential to overstate the effect of outliers or other noise.

Yes, states with similar demographics and economies/economic mixes will likely shift together. But say AZ and NM... there isn't really any pattern as to their results over the years.

I'm not sure what you think the 538 model does, but my understanding is that it uses the actual covariances between all the states over many election cycles in order to determine the proper correlation values. So, if NM and AZ have not historically shifted together, 538's model would not use an AZ poll to affect NM's forecast. The main way this method could fail is if certain states that have shifted together in the past do not shift together this cycle, but I think this would not be that big of an effect in the grand scheme of things.

[Mallow]

I just think you have more faith in the model than I do.

He's too busy covering his arse and so it's far more skittish and prone to big jumps off wonky data points.

It's actually a very common statistical method, used in many fields (including my own) to get the most accurate "snapshot" given sparse data prone to error. It's well-studied and there's plenty of reason to have "faith" in the method.

This whole idea that the model is being somehow manipulated in real-time to suit the preconceptions of Silver et al. is silly and misinformed. The "skittishness" of the model is being overstated, too--the percentage chance for Clinton to win (n.b. NOT the change in the margin) in the "polls-plus" version of the model has changed just 13% in the past three weeks. That's the "big" change we're discussing here. And it's been driven by lots and lots of data, not just one or two data points.

[Mallow]

I think his model is too volatile compared to 2012, which was not only accurate, but mirrored President Obama's internal polling that always showed him ahead

I'd point out that the polls-plus and the polls-only models both have Clinton ahead still, and only the latter of the two has ever shown a trump lead... and then only 50.1% to 49.9% after the RNC for one day. So basically, they have both shown Clinton ahead the whole time.

[Mallow]

538 inputed a WV poll incorrectly. It has a Just Win poll with Clinton +27 when it should be Trump +27.  What a bizarre mistake to make.

Ya, I saw that too. They've since fixed it, though. Odd... you'd think they'd have scripts to automate those things.

[Mallow]

Odd... you'd think they'd have scripts to automate those things.

How would such a script work?  Search through the PDF of the poll release for the Clinton and Trump numbers?  When every pollster will have poll releases that look different?  It's a lot easier to just type in the numbers manually.

I wasn't thinking a single one for all pollsters, I was thinking tailored to each pollster. But it's probably still easier to enter them manually, as you say.

[Mallow]

Sorry, that page doesn't exist.

[Mallow]

He never thought Rhode Island was a swing state. That one poll showing Clinton up by only 3 (with no other polls showing otherwise) did cause Trump to have over a 10 percent chance of winning Rhode Island for a while, which while seemingly ridiculous given past results there, is probably about right given a lack of any other evidence.

That same pollster, Emerson, now has Clinton up by 20 in their latest poll, thereby "fixing" the Rhode Island odds.

It was 25% on the now-cast. lmao. That's completely embarrassing.

Well I'm glad you know better than everyone else that Trump wouldn't have had a 25% chance to win Rhode Island at that exact moment (which is the variable the Now-Cast "predicts").

Seriously, though... without subjectively manipulating the data, you're going to get a wonky result every once in a while in a state without much polling. What would you have them do, make sh**t up?

[Mallow]

He never thought Rhode Island was a swing state. That one poll showing Clinton up by only 3 (with no other polls showing otherwise) did cause Trump to have over a 10 percent chance of winning Rhode Island for a while, which while seemingly ridiculous given past results there, is probably about right given a lack of any other evidence.

That same pollster, Emerson, now has Clinton up by 20 in their latest poll, thereby "fixing" the Rhode Island odds.

It was 25% on the now-cast. lmao. That's completely embarrassing.

Well I'm glad you know better than everyone else that Trump wouldn't have had a 25% chance to win Rhode Island at that exact moment (which is the variable the Now-Cast "predicts").

Any person with a brain cell knew/knows that, but thanks for being glad for me anyway.

Hillary Clinton had a 70% chance of winning Alabama on August 21st, 2016 at 8:03 AM. Prove me wrong.

Yes, I understand that we're talking in circles. My point is that their model is based on data points only. If you want a model that has subjective "checks," don't use 538.

[Mallow]

Some good discussion in this here thread.

[Mallow]

Decided to make a map of all the states that have never dipped below an 80% chance of the leading candidate winning for more than a week:


Clinton — 176
Trump — 146
tossup* — 216

*Edit: I do not think the grey are tossup, I think they are the not-titanium-[party here] states.

This would probably be an improvement on your map


50% red: Clinton 60% or greater chance to win at all times
30% red: Clinton always in the lead, but under 60% chance to win at one point
Gray: Candidates have exchanged the lead
30% blue: Trump always in the lead, but under 60% chance to win at one point
50% blue: Trump 60% or greater chance at all times

My own take, using the polls-plus version:

Here, tossups are states where the lead has changed. Otherwise, states are colored depending on the lowest percentage that the leader had to win the state (>50%, >60%, >70%, >80%, or >90%, from the lighter colors to the darker colors, respectively).

That's...
Clinton: 268 (176 safe, 63 likely, 29 lean)
Trump: 190 (137 safe, 42 likely, 11 lean)
Tossup: 80

[Mallow]

It's not his fault there have been so many garbage polls released recently -- but his model is supposed to weigh them accordingly based on their rating. That hasn't happened the way it's supposed to because the sheer volume of junk polls is flooding the model in a way that didn't happen in 2012.

At least in pollster.com you can modify which polls are included.

I hate to say this because I'm not liking 538's results lately, but Nate is obviously right here and Grimm is being a pretentious dick.

Guess what? 538's model has always been adjusting for trendline, including in 2008 and 2012 when everybody was in awe at its greatness. You are entitled to believe that it's a faulty method, but cries of "UNSKEWING!!!!!!!!" make no sense.

As has been said, the real problem with 538 isn't the trendline adjustment, it's that it pays too much attention to crappy pollsters.

Yep.

[Mallow]

lol at the conspiracy theories.