RCP rehabilitates ARG, pisses off 538

[Angel of Death]

http://www.fivethirtyeight.com/2008/09/real-credibility-problems.html
http://www.fivethirtyeight.com/2008/10/rcp-follow-up.html

[Sam Spade]

What I'm still trying to figure out is why Insider Advantage is in their average...

[Alcon]

RCP's awful lack of transparency has been pretty clear to anyone paying attention since 2003ish.

[Small Business Owner of Any Repute]

RCP's awful lack of transparency has been pretty clear to anyone paying attention since 2003ish.

Their bias was evident even in 2000.

[Lief 🗽]

Pollster's website is also prettier.

[Lunar]

Pollster.com has become amazing.

Everyone at work is really amused by the 538-RCP feud.

[Eraserhead]

RCP is dominated by right-wingers. I've actually talked to one of the few Democrats who works there before. Poor guy.

[Mr. Morden]

I agree that it makes no sense to purge ARG from the database, and then reinstate it once it starts showing results that you like.  However, I'd like to speak out in favor of subjectivity with regard to choosing which pollsters to include.  Nate Silver implies that RCP is doing something illegitimate by disregarding certain pollsters based on fuzzy criteria that aren't spelled out anywhere.  But is that necessarily a bad thing?

Recall how most of us on this forum considered ARG a joke poll *even before the voting started in this year's primaries*.  This was not because of any past record of inaccuracy.  It was because ARG's polling for the primaries was so wildly off from every other poll in existence.  It was only once the voting started in the primaries that we had definitive proof that ARG was a joke, but we pretty much knew beforehand.  ARG's final polls before an election have always been bad, but their polls taken weeks/months before the election are typically much worse, and this is not reflected in anyone's "pollster ratings" because one typically only considers the accuracy of the final poll taken before the election.

Additionally, ARG's polling seemed to magically improve after Super Tuesday.  I doubt this is because they suddenly became much better pollsters.  More likely, they just started copying more credible polls.  But there's no way to prove this, and no way to score that in any kind of objective analysis of their accuracy.  So I think excluding them from a poll average for no other reason than that your gut tells you that they're faking it (and not because of any objective criteria) is a defensible position.  But yeah, once you do that, stick with the decision.  Don't renege on it for no apparent reason.

[Lunar]

no defense of ARG or Zogby is legitimate.

[Likely Voter]

RCP's selective polling has become an issue, especially in states with fewer polls. There doesn't seem to be a logic to it. Pollster now allows you to filter polls and even change the trendline. The problem with both is that as national polls move in one direction, they still only have state polls to average. That is why 538 has become popular as it adds demographics and national poll numbers as well as pollster weightings.

Many states have few or even no polls since the recent economic problems...so 538 is the only one that is at least taking a stab. Of course they could be dead wrong

[Mr. Morden]

no defense of ARG or Zogby is legitimate.

Wait, I'm confused.  Was that a response to my post?  Because I wasn't defending ARG.  I was doing the opposite.  I was explaining why it's legitimate to exclude ARG while keeping other pollsters that rank worse in "pollster accuracy", because all of the "pollster accuracy" scorecards miss certain clues that tell you that ARG is fraudulent.

[minionofmidas]

no defense of ARG or Zogby is legitimate.

Wait, I'm confused.  Was that a response to my post?  Because I wasn't defending ARG.  I was doing the opposite.  I was explaining why it's legitimate to exclude ARG while keeping other pollsters that rank worse in "pollster accuracy", because all of the "pollster accuracy" scorecards miss certain clues that tell you that ARG is fraudulent.

You were defending RCP's defense of ARG.

At least, I think that's what Lunar was aiming at.

[Mr. Morden]

no defense of ARG or Zogby is legitimate.

Wait, I'm confused.  Was that a response to my post?  Because I wasn't defending ARG.  I was doing the opposite.  I was explaining why it's legitimate to exclude ARG while keeping other pollsters that rank worse in "pollster accuracy", because all of the "pollster accuracy" scorecards miss certain clues that tell you that ARG is fraudulent.

You were defending RCP's defense of ARG.

Not really, no.  I was defending the general concept of using somewhat subjective criteria to decide which pollsters to include in averages.  The specific example I was using to explain this was a defense of RCP's original decision to exclude ARG.  Nate Silver thinks ARG shouldn't be excluded because there are other pollsters who are "worse".  I questioned whether ARG's badness can really be captured in the criteria that Silver would use.  That's roughly the gist of what I said.

[minionofmidas]

Yeah, I know.

[classical liberal]

no defense of ARG or Zogby is legitimate.

[Queen Mum Inks.LWC]

But, RCP is where I find out about a lot of the polls (other than Strat Vis, SUSA, and Ras), although I never paid attention to the averages there.

[The Vorlon]

My big beef with ARG is they essentially just throw out a number.

In 2004 they had a pretty pro democratic bias, if anything this year they slant the other way.

Why?

Did they change their methodology, are they using some hard weight we don't know about...?

Did they just say "in 2004 we sucked, and on average underpolled the GOP by 5%, so we will just do everything the same, except add 5% to McCain when we release it..."...?

I understand how Gallup works, I understand how Rasmussen works, I understand how Democracy Corps, Battleground, SUSA, PEW, ABC, and Mason Dixon work - I can draw intelligent conclusions from what they publish even when their numbers differ from one another.

ARG and Zobgy...?

They may as well be pulling numbers out of a hat from what I can tell

(Actually I believe Zogby uses dice, a ouiji board, animal bones and tarrot cards, and channelling the dead for the online versioins, but that is just a personal theory)

[Mr. Morden]

ARG's methodology is actually quite easy to understand.  They use this formula.