2018 Congressional Generic Ballot and House Polls Megathread - Part 3

[Devils30]

Siena polls also tend to favor incumbents in September and then break sharply toward challengers in October.

[Brittain33]

FL-6 St Pete Polls: Waltz (R) 48, Soderberg (D) 43

http://floridapolitics.com/archives/275276-waltz-soderberg-cd6-poll

Yeah she gonna lose.

This is not a district that has voted for a Dem in my adult lifetime.

Nelson won it pretty easily in 2012

Apologies, I should have specified for House of Representatives. Although it is helpful to note that it's not a solid R district like FL-1.

[Fmr. Gov. NickG]

https://twitter.com/MonmouthPoll/status/1042790579900440577?s=19

So do we know if Malinowski himself responded to this poll or the NYT/Siena one?
That could at least explain 0.2% of the difference.

[Zaybay]

Siena polls also tend to favor incumbents in September and then break sharply toward challengers in October.

This, Siena usually gives the benefit of the doubt to incumbents in these months, and only in October do they swing. This was the same in 2010, and 2014, except against Democrats.

[Xing]

Is there historical evidence to back up the idea that Siena is biased toward incumbents in September? It's certainly not the case that all of their polls are rosy for incumbents.

[Politician]

Is there historical evidence to back up the idea that Siena is biased toward incumbents in September? It's certainly not the case that all of their polls are rosy for incumbents.

It's just a lie #BlueWavers tell themselves to feel better about those brutal numbers they've been seeing lately.

Erik Paulsen, Mike Coffman and Rod Blum say hi.

Also, incumbents at 45% is bad news for them, especially when undecideds lean Democratic.

[GeorgiaModerate]

https://twitter.com/MonmouthPoll/status/1042790579900440577?s=19

So do we know if Malinowski himself responded to this poll or the NYT/Siena one?
That could at least explain 0.2% of the difference.

Nate Cohn responded to Malinowski's tweet saying that it was not the NYT one.

[Wisconsin SC Race 2019]

Is there historical evidence to back up the idea that Siena is biased toward incumbents in September? It's certainly not the case that all of their polls are rosy for incumbents.

It's just a lie #BlueWavers tell themselves to feel better about those brutal numbers they've been seeing lately.

Erik Paulsen, Mike Coffman and Rod Blum say hi.

Also, incumbents at 45% is bad news for them, especially when undecideds lean Democratic.

Sure, those 3 are DOA, but we're gonna need 22 more to take the House. Where are they gonna come from? All these "magical Latino wave" seats that Dems have sacrificed the rurals on an altar for aren't happening. The Tejanos and Cubans are sticking with the GOP for now.

[Virginiá]

Is there historical evidence to back up the idea that Siena is biased toward incumbents in September? It's certainly not the case that all of their polls are rosy for incumbents.

It's just a lie #BlueWavers tell themselves to feel better about those brutal numbers they've been seeing lately.

Erik Paulsen, Mike Coffman and Rod Blum say hi.

Also, incumbents at 45% is bad news for them, especially when undecideds lean Democratic.

Sure, those 3 are DOA, but we're gonna need 22 more to take the House. Where are they gonna come from? All these "magical Latino wave" seats that Dems have sacrificed the rurals on an altar for aren't happening. The Tejanos and Cubans are sticking with the GOP for now.

Do you even pay attention to House ratings? Most seats Democrats are after arguably don't rely on Latinos that much or have enough other factors that are making the relative Latino indifference in this election not a major problem. Even for the seats where Latinos could prove an issue, there are so many other potential pickups that could work out.

Once again, you don't know what you are talking about.

[BudgieForce]

Both NJ-7 polls have small samples, so both polls would be within each others margin of error, right? Or am I making that up?

Edit: oh, Nate Silver already answered my question.

https://twitter.com/NateSilver538/status/1042794668553986048

[Xing]

Thanks for the low effort, predictable "bloo wave is ded kek" troll response. Much obliged.

What I was asking is if Siena polls from other cycles have overestimated an incumbent's eventual margin. 45% for an incumbent isn't great, but have Siena polls actually said that, for example, that an incumbent who was up around 46-43 in September ended up losing?

[GeorgiaModerate]

Both NJ-7 polls have small samples, so both polls would be within each others margin of error, right? Or am I making that up?

You're correct.  Monmouth (standard model) has it 47-39 Malinowski with a MoE of about 5.  So Malinowski is within 42-52 and Lance 34-44, 19 times out of 20.  NYT has Lance 48-44 with a MoE of 6, which would mean Lance 42-54 and Malinowski 38-50.   

Lance: 42-54 and 34-44 (overlap 42-44).
Malinowski: 38-50 and 42-54 (overlap 42-50).

[BudgieForce]

Both NJ-7 polls have small samples, so both polls would be within each others margin of error, right? Or am I making that up?

You're correct.  Monmouth (standard model) has it 47-39 Malinowski with a MoE of about 5.  So Malinowski is within 42-52 and Lance 34-44, 19 times out of 20.  NYT has Lance 48-44 with a MoE of 6, which would mean Lance 42-54 and Malinowski 38-50.   

Lance: 42-54 and 34-44 (overlap 42-44).
Malinowski: 38-50 and 42-54 (overlap 42-50).

Thank you. I think alot of poll watchers dont really get the concept of MoE(I certainly dont). So neither the Sienna and Monmouth polls are out of left field.

[Zaybay]

Is there historical evidence to back up the idea that Siena is biased toward incumbents in September? It's certainly not the case that all of their polls are rosy for incumbents.

Yes actually.

https://scri.siena.edu/category/political/page/25/

This is the section that has most of the 2010 polls. As you can see, some of these results gathered before October are rather crazy:
-the 24th had the R, Richard Hanna, down by 8, 48-40. A poll released in the last week had a more modest win for Democratic incumbent, Mike Arcuri, of 48-43. The R won 52-48.

-In the 20th, Chris Gibbson won 55-45, Siena gave the incumbent D, Scott Murphy the lead, by 17 points no less

-One of the worst offenders, Siena gave, in October, the incumbent Democrat Dan Maffei a large lead in the 25th district, 51-39. The republican challenger, Ann Marie Buerkle, won by .5%

(Siena did NY polls most of the time)

2014 is similar
-Tim Bishop, the incumbent, was given a 51-41 lead against challenger Lee Zeldin, halfway through September. Siena later corrected this in the last week of the election, giving Zeldin a 5% lead. He won  53-44.

-A similar story, Siena gave incumbent Dan Maffei a lead, 50-42, in NY-24. Katko got a more favorable poll, 52-42, in the last week of the election, and won 58-38.

Even in 2018, there has been a similar trend, with its NY polls having a much more Republican sample, and finding much higher approvals than, say Monmouth, or other pollsters have found. There is definitely an incumbency bias in Siena.

Edit: this took a while to get, so I apologize for not getting it sooner

[Virginiá]

Sabato's ratings so far (current count, not individual):

http://crystalball.centerforpolitics.org/crystalball/articles/ratings-changes-house-and-governors-2018-09-19/

You must be logged in to read this quote.

and:

You must be logged in to read this quote.

Democrats are pretty close a majority in his ratings.

[TheRocketRaccoon]

https://www.charlotteobserver.com/news/politics-government/election/article218705250.html

NC-13 DCCC (D internal): Manning (D) 46, Budd (R-inc) 42

[GeorgiaModerate]

Both NJ-7 polls have small samples, so both polls would be within each others margin of error, right? Or am I making that up?

You're correct.  Monmouth (standard model) has it 47-39 Malinowski with a MoE of about 5.  So Malinowski is within 42-52 and Lance 34-44, 19 times out of 20.  NYT has Lance 48-44 with a MoE of 6, which would mean Lance 42-54 and Malinowski 38-50.  

Lance: 42-54 and 34-44 (overlap 42-44).
Malinowski: 38-50 and 42-54 (overlap 42-50).

Thank you. I think alot of poll watchers dont really get the concept of MoE(I certainly dont). So neither the Sienna and Monmouth polls are out of left field.

I think that's true.  Most people want answers to simple binary questions like "who's going to win?" or at least "who's ahead right now?"  Political polls don't answer those questions; they measure how many people say they'll vote for a particular candidate, which is not the same thing, for several reasons:

1. What people say may not match who they actually vote for.
2. Margin of error as a result of sampling.
3. Outliers (even with a carefully designed poll, you'll get an outlier outside MoE 5% of the time).

So at best, polls provide somewhat fuzzy estimates of support.  But what we really want to know is who's going to win.  It used to irritate me when people would say pollster X got a race "wrong" when they had candidate A ahead by 2 (with a MoE of say 4), but A lost by 1. Meanwhile, they'd say a poll that had B ahead by 15 was "right", because they're only looking at the binary outcome.

Pollsters are continually trying to refine and improve their methodology, but no matter how good they get, they will never be able to generate precise measurements of the outcome.  This is inherent in the nature of sampling and statistics.  The cliche that "the only poll that matters is the one on Election Day" is actually true -- or at least it's the only poll that's truly accurate.  A poll getting very close to a final result includes an element of luck, no matter how good the pollster is.

It's understandable that people just want a simple answer of who's going to win.  That's human nature, and I'm not immune to it.  All we can really do is try and remember what they can tell us, and what they can't.

[Florida Man for Crime]

Is there historical evidence to back up the idea that Siena is biased toward incumbents in September? It's certainly not the case that all of their polls are rosy for incumbents.

Yes, I have heard this claim repeated often. What I have not seen, though, is a serious/comprehensive analysis that actually establishes that as a fact.

Nor moreover, have I seen anything that establishes that if it is true, it is something that specifically truly applies just to Siena specifically. It is plausible that this could be true but also apply to other polls, if it is simply a reflection of the fact that challengers have lower name recognition than incumbents until close to the election.

Also, incumbents at 45% is bad news for them, especially when undecideds lean Democratic.

This is true and a good caveat to the above (although we cannot know with any certainty in advance how undecideds will break in any given race). We have seen lots of polls with incumbents at around 45% and sometimes even below, and those are definitely bad news for GOP incumbents, even if they are nominally ahead.

[Ebsy]

To be noted, Cohn is not using Sienna for all of these polls. I think there are 4 or 5 other call centers scattered around the country.

[Co-Chair Bagel23]

https://www.charlotteobserver.com/news/politics-government/election/article218705250.html

NC-13 DCCC (D internal): Manning (D) 46, Budd (R-inc) 42

Nice.

[Florida Man for Crime]

Is there historical evidence to back up the idea that Siena is biased toward incumbents in September? It's certainly not the case that all of their polls are rosy for incumbents.

Yes actually.

https://scri.siena.edu/category/political/page/25/

This is the section that has most of the 2010 polls. As you can see, some of these results gathered before October are rather crazy:
-the 24th had the R, Richard Hanna, down by 8, 48-40. A poll released in the last week had a more modest win for Democratic incumbent, Mike Arcuri, of 48-43. The R won 52-48.

-In the 20th, Chris Gibbson won 55-45, Siena gave the incumbent D, Scott Murphy the lead, by 17 points no less

-One of the worst offenders, Siena gave, in October, the incumbent Democrat Dan Maffei a large lead in the 25th district, 51-39. The republican challenger, Ann Marie Buerkle, won by .5%

(Siena did NY polls most of the time)

2014 is similar
-Tim Bishop, the incumbent, was given a 51-41 lead against challenger Lee Zeldin, halfway through September. Siena later corrected this in the last week of the election, giving Zeldin a 5% lead. He won  53-44.

-A similar story, Siena gave incumbent Dan Maffei a lead, 50-42, in NY-24. Katko got a more favorable poll, 52-42, in the last week of the election, and won 58-38.

Even in 2018, there has been a similar trend, with its NY polls having a much more Republican sample, and finding much higher approvals than, say Monmouth, or other pollsters have found. There is definitely an incumbency bias in Siena.

Edit: this took a while to get, so I apologize for not getting it sooner

So just eyeballing this quickly, you have a sample size of 5 polls you are working with here, and only over 2 elections. That is not a large amount of data from which to draw valid conclusions.

And for this analysis you seem to be selecting polls that had large differences between the poll and the results, but ignoring other polls.

In addition, all of the polls you mention were in New York. An alternate explanation (or partial explanation) for sharp divergences/swings could be that New York is simply very swingy and elastic, particularly upstate. This would definitely be consistent with actual election results, where there have been very sharp differences in results between different years and in different races. Democrats have often carried upstate NY counties in landslides (especially incumbent dems in statewide races), but on the other hand they have also voted in landslides for Republicans (Trump being a recent case in point, but also earlier Republicans have done so). In fact, if you had to pick out a single region of the United States as the most swingy/elastic, upstate NY would be a prime contender.

And even if your analysis is correct, in order for it to say anything particular about Siena, you need to compare it to other pollsters to show that it is actually a Siena-specific effect.

[GeorgiaModerate]

To be noted, Cohn is not using Sienna for all of these polls. I think there are 4 or 5 other call centers scattered around the country.

In addition, he's said that they're using Upshot's own weighting choices, not Siena's.

[Florida Man for Crime]

To be noted, Cohn is not using Sienna for all of these polls. I think there are 4 or 5 other call centers scattered around the country.

Call center is not the same as pollster. Pollsters can contract out a call center to conduct the actual calls, but still they set up the poll, questions, analyze the data, etc.

[Florida Man for Crime]

Most people want answers to simple binary questions like "who's going to win?" or at least "who's ahead right now?"  Political polls don't answer those questions; they measure how many people say they'll vote for a particular candidate, which is not the same thing, for several reasons:

1. What people say may not match who they actually vote for.
2. Margin of error as a result of sampling.
3. Outliers (even with a carefully designed poll, you'll get an outlier outside MoE 5% of the time).

Good post. To the 3 reasons you posted, I would propose adding a few more:

4. By definition of "undecided," polls do not measure who undecided voters will vote for (if indeed they do vote).
5. Even if everyone in a poll votes for who they say they will, and there is no statistical error, the weighting/proportions of people included in the poll may not match the weighting/proportions of people who actually turn out to vote.
6. Polls ask the question of who you will vote for in a different way than a ballot does - a ballot may (or may not) have straight party voting, and lists any 3rd party candidates and write-ins in a particular way, which cannot be replicated by a pollster either including or not including 3rd party candidates as explicit options in their poll questions.

[GeorgiaModerate]

To be noted, Cohn is not using Sienna for all of these polls. I think there are 4 or 5 other call centers scattered around the country.

Call center is not the same as pollster. Pollsters can contract out a call center to conduct the actual calls, but still they set up the poll, questions, analyze the data, etc.

I believe Siena is only fulfilling the call center function (as are some other call centers), and the question design, etc., is coming from NYT/Upshot.  See the tweet below.  If this is the case, all the arguments about Siena's supposed incumbent bias are not relevant to these polls.

https://twitter.com/Nate_Cohn/status/1041806028822720512