According to Nate Silver, PPP not even close to being the best pollster of 2012

[Tender Branson]

5 polls minimum:



1 poll minimum:



http://fivethirtyeight.blogs.nytimes.com/2012/11/10/which-polls-fared-best-and-worst-in-the-2012-presidential-race

[Brittain33]

PPP has a bias of 1.6 points. For Republicans.

J.J., does this claim stand uncontested, or will you debunk it in favor of the 3 point Dem bias you documented?

[Comrade Funk]

Initiate J.J effect....

PPP has a Democratic bias of 1.4 when you add the 3 point Democratic house effect.

[J. J.]

There are some obvious problems with Silver's methodology.  The last 21 days does not take into account any last minute swings.  It also doesn't take into account that some of the pollsters didn't poll due to Sandy.

[Franzl]

There are some obvious problems with Silver's methodology.  The last 21 days does not take into account any last minute swings.  It also doesn't take into account that some of the pollsters didn't poll due to Sandy.

Do you maintain your claim of a 3 point or more "house bias" in PPP polling?

[Skill and Chance]

lol at Marist having an R+1 house effect.

[Mehmentum]

LOL Gallup.  Its a full 3 points worse than the next worse poll, ARG. 

[Comrade Funk]

Remember when Republican after Republican kept whining that all polls were bias towards Democrats? Looks like the complete opposite happened.

[Brittain33]

Remember when Republican after Republican kept whining that all polls were bias towards Democrats? Looks like the complete opposite happened.

I have to cut them some slack... I thought they were trolling or willfully propagandizing for their candidate... But all the news coming out reports they really believed that the election was going to look like 2004. So they were wrong, but they were honestly wrong, and with one notable exception here they have accepted that.

[Speed of Sound]

Yeah, that last national poll used to determined the "accuracy" in the other report was really generous, this is much more interesting and telling.

[Yank2133]

Remember when Republican after Republican kept whining that all polls were bias towards Democrats? Looks like the complete opposite happened.

Yup.

[Kitteh]

Gallup was never a very good pollster. Their credibility pretty much comes from being the first major polling organization. They're just riding off their legacy at this point.

[bgwah]

I'm pretty sure PPP's ridiculous polls near the end showing small leads for Obama in states like WA and OR were specifically intended to make it look like they don't have a Democratic bias.

[Badger]

Looks like we better start considering PPP's (R) house bias when reviewing their polls.

[Badger]

I'm pretty sure PPP's ridiculous polls near the end showing small leads for Obama in states like WA and OR were specifically intended to make it look like they don't have a Democratic bias.

Maybe. Though is it just me or it seems like a pollster will be off on a whole set of polls they do together the same day, even in different states? In other words, regardless of what general house effect a pollster might have, if they do a poll in (e.g.) Oregon that seems to lean R, a poll they do/release the same day for (e.g.) WA will tend to have the same R lean? I noticed this several times throughout the campaign with many pollsters. I meant to start a thread about it, but never got around to doing so.

[Lief 🗽]

I have some problems with Silver's methodology here. First, they're not actually done counting all the votes. There are still a few million votes nationally to count, so the margin will change there. The margin is also likely to change by around a point in at least a few states (Arizona, Ohio). Not sure why he couldn't have waited another week or so. Second, I'm not sure how you can compare polls three weeks before the election to the actual election result for accuracy. Part of the reason why I suspect every poll (almost) has a Republican "house effect" here is because Mitt Romney was doing a lot better in the polls three weeks before the election than he was a day before the election. You have no way of knowing if a poll weeks before an election is accurate. There's nothing to test it against. It makes more sense to just use the last poll nearest to the election.

[King]

Come on now. Do we really think TIPP would keep that up if they did 71 polls? The comparison should just be between the only volume pollsters.

PPP's model bested Rasmussen.  Online pollsters Ipsos and YouGov bested them both.

[minionofmidas]

I've always had a problem with using only final polls. You need to evaluate all their polling - you'll just have to compare with other polls with similar timestamps to correct for actual changes in the race.

[GMantis]

These results clearly show that from now on cell phones will have to be included in samples. This however makes polling far more expensive, so we can probably expect ever less polling of safe states (including "safe" states, as defined by companies like Suffolk) in the future.

[Franzl]

There are some obvious problems with Silver's methodology.  The last 21 days does not take into account any last minute swings.  It also doesn't take into account that some of the pollsters didn't poll due to Sandy.

Do you maintain your claim of a 3 point or more "house bias" in PPP polling?

[Zanas]

I'm with Lief on that one. You cannot just aggregate all the polls done in the last 21 days of campaign and compare them to the actual result to test for a house bias. It doesn't make sense.

I think the most reliable way to test for house bias is to aggregate all polls done for a same area in the same week (and preferably paying attention to huge events like first debate, Sandy, et al.) and testing each with the mean of all. That way you have them polling the same electorate at the same time and you can effectively test for biases. Otherwise it's just sh**t.

But in Nate we Trust, so I'm sure he will do just that when all the final results are in.

[🐒Gods of Prosperity🔱🐲💸]

There are some obvious problems with Silver's methodology.  The last 21 days does not take into account any last minute swings.  It also doesn't take into account that some of the pollsters didn't poll due to Sandy.

Do you maintain your claim of a 3 point or more "house bias" in PPP polling?

Why don't you ask Nate Silver since he's the one who came up with that number.
http://fivethirtyeight.blogs.nytimes.com/2012/06/22/calculating-house-effects-of-polling-firms/

[Tender Branson]

There should be 1 examination using only the final polls from each pollster and those should not be older than 1 week before election day.

Another examination should focus how the different pollsters did before that 1 week timespan.

[I spent the winter writing songs about getting better]

There are some obvious problems with Silver's methodology.  The last 21 days does not take into account any last minute swings.  It also doesn't take into account that some of the pollsters didn't poll due to Sandy.

Do you maintain your claim of a 3 point or more "house bias" in PPP polling?

Why don't you ask Nate Silver since he's the one who came up with that number.
http://fivethirtyeight.blogs.nytimes.com/2012/06/22/calculating-house-effects-of-polling-firms/

...in June. And with a model that compared to the other pollsters and the average, not the actual result for obvious reasons.

J. J. seems to be sticking to his guns that PPP has a Dem bias. It's just like the Bradley Effect in 2008 all over again.

[muon2]

PPP has a bias of 1.6 points. For Republicans.

J.J., does this claim stand uncontested, or will you debunk it in favor of the 3 point Dem bias you documented?

On the flip side a lot of Dems here were busting WAA, and it turns out they had no statistical bias at all (D+0.1).