Nate Silver calculates 2012 house effects of different pollsters.

[tpfkaw]

http://fivethirtyeight.blogs.nytimes.com/2012/06/22/calculating-house-effects-of-polling-firms/

[The Mikado]

So CNN's the least partisan (not necessarily most accurate, mind)?

[Bandit3 the Worker]

I stopped reading when he said SurveyUSA has a Democratic bias.

[tpfkaw]

So CNN's the least partisan (not necessarily most accurate, mind)?

According to Silver, yes.  (Although that's a good demonstration about how merely not having a house effect doesn't mean you aren't a crap pollster).

[Supersonic]

PPP's house effect is far higher than Rasmussen. Interesting. Looking around this forum, I would have thought it to be the other way around. I think PPP polls should be taken with a pinch of salt.

[muon2]

PPP's house effect is far higher than Rasmussen. Interesting. Looking around this forum, I would have thought it to be the other way around. I think PPP polls should be taken with a pinch of salt.

Silver explains that in the article. Rassy uses likely voters which are inherently more GOP, but also more accurate near the election. PPP and others have been mostly turning in registered voter polls, and he notes that he makes a correction for that when he uses them in his projections.

[Bacon King]

One thing to keep in mind is that this is just the average difference between a pollster's result and the average of all polls, not that every poll from a firm should be automatically assumed to be X% more Democratic or Republican to be accurate. For example, the generic congressional ballot polls in 2006 had a fairly wide variation, but the actual result wasn't anywhere near the average (it was actually several points more Democratic than the most left-leaning major tracking poll).

[krazen1211]

One thing to keep in mind is that this is just the average difference between a pollster's result and the average of all polls, not that every poll from a firm should be automatically assumed to be X% more Democratic or Republican to be accurate. For example, the generic congressional ballot polls in 2006 had a fairly wide variation, but the actual result wasn't anywhere near the average (it was actually several points more Democratic than the most left-leaning major tracking poll).

True. Thus far, PPP is a mere 3 points off the polling average and 4-5 off of reality.

[True Federalist (진정한 연방 주의자)]

One thing to keep in mind is that this is just the average difference between a pollster's result and the average of all polls, not that every poll from a firm should be automatically assumed to be X% more Democratic or Republican to be accurate. For example, the generic congressional ballot polls in 2006 had a fairly wide variation, but the actual result wasn't anywhere near the average (it was actually several points more Democratic than the most left-leaning major tracking poll).

True. Thus far, PPP is a mere 3 points off the polling average and 4-5 off of reality.

No need to be so pessimistic.  Obama won't do 5 to 10 points better than the pollsters say he will.

[Vosem]

One thing to keep in mind is that this is just the average difference between a pollster's result and the average of all polls, not that every poll from a firm should be automatically assumed to be X% more Democratic or Republican to be accurate. For example, the generic congressional ballot polls in 2006 had a fairly wide variation, but the actual result wasn't anywhere near the average (it was actually several points more Democratic than the most left-leaning major tracking poll).

True. Thus far, PPP is a mere 3 points off the polling average and 4-5 off of reality.

No need to be so pessimistic.  Obama won't do 5 to 10 points better than the pollsters say he will.

True. But he may (not necessarily will, but may) do 5-10 points worse.

[WhyteRain]

http://fivethirtyeight.blogs.nytimes.com/2012/06/22/calculating-house-effects-of-polling-firms/

That's not much of a spread -- from +3.2 to -2.5 for the Democrats.  I'm not saying it's inaccurate.  I never expected the Democrats to win the House in 2012.

Did Silver do a similar calculation for the Senate races?  I predict a result between -3 and -7 for the Democrats (a really wide difference, I know).

[Torie]

http://fivethirtyeight.blogs.nytimes.com/2012/06/22/calculating-house-effects-of-polling-firms/

That's not much of a spread -- from +3.2 to -2.5 for the Democrats.  I'm not saying it's inaccurate.  I never expected the Democrats to win the House in 2012.

Did Silver do a similar calculation for the Senate races?  I predict a result between -3 and -7 for the Democrats (a really wide difference, I know).

You do know that "house effects" is not about the House of Representatives, right?

[WhyteRain]

http://fivethirtyeight.blogs.nytimes.com/2012/06/22/calculating-house-effects-of-polling-firms/

That's not much of a spread -- from +3.2 to -2.5 for the Democrats.  I'm not saying it's inaccurate.  I never expected the Democrats to win the House in 2012.

Did Silver do a similar calculation for the Senate races?  I predict a result between -3 and -7 for the Democrats (a really wide difference, I know).

You do know that "house effects" is not about the House of Representatives, right?

I do now.  My bad.

[greenforest32]

One thing to keep in mind is that this is just the average difference between a pollster's result and the average of all polls, not that every poll from a firm should be automatically assumed to be X% more Democratic or Republican to be accurate. For example, the generic congressional ballot polls in 2006 had a fairly wide variation, but the actual result wasn't anywhere near the average (it was actually several points more Democratic than the most left-leaning major tracking poll).

Yeah it will be interesting to see how the 'house effect' difference compares to the results difference. Did he do a 'house effect' calculation in 2008 or 2010? I could only find comparisons to the actual results in 2010. I have a hard time believing Quinnipiac is more biased to Republicans than Ras.

http://fivethirtyeight.blogs.nytimes.com/2010/11/04/rasmussen-polls-were-biased-and-inaccurate-quinnipiac-surveyusa-performed-strongly/

[Alcon]

I think we overestimate the extent to which pollsters can really control their house effects.  Keep in mind that these poor souls are dealing with record-low successful contact rates (like 10%) and it's pretty amazing that they still manage what they do.  Scott Rasmussen is obviously a Republican, but that doesn't mean that Scott Rasmussen is an evil scientist with a staff of corrupt statisticians bent on making their jobs harder by calibrating weights and likely voter models to intentionally add subtle bias.

[Brittain33]

One thing to keep in mind is that this is just the average difference between a pollster's result and the average of all polls, not that every poll from a firm should be automatically assumed to be X% more Democratic or Republican to be accurate. For example, the generic congressional ballot polls in 2006 had a fairly wide variation, but the actual result wasn't anywhere near the average (it was actually several points more Democratic than the most left-leaning major tracking poll).

True. Thus far, PPP is a mere 3 points off the polling average and 4-5 off of reality.

No need to be so pessimistic.  Obama won't do 5 to 10 points better than the pollsters say he will.

True. But he may (not necessarily will, but may) do 5-10 points worse.

Your post is in the style and format of what J.J. would say.

[True Federalist (진정한 연방 주의자)]

I think we overestimate the extent to which pollsters can really control their house effects.  Keep in mind that these poor souls are dealing with record-low successful contact rates (like 10%) and it's pretty amazing that they still manage what they do.  Scott Rasmussen is obviously a Republican, but that doesn't mean that Scott Rasmussen is an evil scientist with a staff of corrupt statisticians bent on making their jobs harder by calibrating weights and likely voter models to intentionally add subtle bias.

True, but since gaging what be the actual voter composition requires a pollster to make assumptions, it is not surprising that when making assumptions pollsters will tend to unintentionally make the electorate a bit more like them than it actually is.