Are the polls skewed?

[mondale84]

Now Chris Wallace is saying attacking the polls is "crazy"
http://www.mediaite.com/tv/chris-wallace-snaps-at-right-wing-host-criticizing-polls-that-show-romney-in-trouble-is-craziness/

LOL. Even the right-wing hacks see they're doomed.

[Likely Voter]

Mark Blumenthal has a great post on this whole skewed thing and it does a good job of explaining how polls actually work. He also draws parallel with 2004, when Dems were griping about Party ID too

http://www.huffingtonpost.com/mark-blumenthal/unskewed-polls_b_1924293.html?utm_hp_ref=@pollster

[Torie]

Here is the problem. Cell phone folks, and persons of color and so forth don't answer the phone as much as old white females (yes women answer the phone more). So you have to re-weight your "random" poll result subtotals to get the demographics "right."  And therein lies a problem. How do you know that folks within a demographic group break the same between those with land lines and those without? Or folks who respond to polls and those who hang up the phone? Or just how much of the subgroup will show up to vote, particularly the cell phone folks, but also when stated intention to vote does not match previous behavior for a subgroup. It is a particularly serious problem with Hispanics. They must be rather massively under-polled, and need a large re-weight.

I note Blumenthal says they just re-weight to census data. OK, whatever. That seems rather crude to me. The same issue obtains. There is no escape.

So if after you create your demographic turnout model (based on the past presumably), and you get a sample that is has say a Dem margin of 13%, maybe you should start to worry. So it is not quite as simple as stating that those who re-weight based on partisan affiliation, are just hopeless, clueless, innumerate knuckle draggers who slept through their statistics course, or never bothered to take one. It all depends. And to add to the stress, the demographics have become more volatile from 2004 to 2006 to 2008 to 2010, bouncing, bouncing all around a bit - particularly as the young and the black found their pied piper, and their turnout went up, up and away.

And the truth is that every election is so unique, and the data points so few anyway, that this demographic re-weighting stuff just has to be more art than science.

Oh, to answer the poll:  I don't know!

[Likely Voter]

In 2004 after a successful RNC and lackluster DNC (with candidate to match) there was a surge in polling showing more R identification. In 2012 the opposite is happening, with the added boost of the 47% thing.

It's Occam's Razor. The simplest solution is that in the last month more people have identified with the Dems.  There is no reason to believe that in the last month polling "broke" or even worse, there was a conspiracy.

The fundamental fallacy is to view party ID as the same as party registration or even something like sex or age. If you think of it like presidential approval or right track/wrong track (which is more apt) then it all makes sense.  

[ajb]

Here is the problem. Cell phone folks, and persons of color and so forth don't answer the phone as much as old white females (yes women answer the phone more). So you have to re-weight your "random" poll result subtotals to get the demographics "right."  And therein lies a problem. How do you know that folks within a demographic group break the same between those with land lines and those without? Or folks who respond to polls and those who hang up the phone? Or just how much of the subgroup will show up to vote, particularly the cell phone folks, but also when stated intention to vote does not match previous behavior for a subgroup. It is a particularly serious problem with Hispanics. They must be rather massively under-polled, and need a large re-weight.

I note Blumenthal says they just re-weight to census data. OK, whatever. That seems rather crude to me. The same issue obtains. There is no escape.

So if after you create your demographic turnout model (based on the past presumably), and you get a sample that is has say a Dem margin of 13%, maybe you should start to worry. So it is not quite as simple as stating that those who re-weight based on partisan affiliation, are just hopeless, clueless, innumerate knuckle draggers who slept through their statistics course, or never bothered to take one. It all depends. And to add to the stress, the demographics have become more volatile from 2004 to 2006 to 2008 to 2010, bouncing, bouncing all around a bit - particularly as the young and the black found their pied piper, and their turnout went up, up and away.

And the truth is that every election is so unique, and the data points so few anyway, that this demographic re-weighting stuff just has to be more art than science.

Oh, to answer the poll:  I don't know!

I think you're missing the point of part of what Blumenthal is saying, which is that reputable pollsters don't have a "demographic turnout model." To quote from the piece:

" They first sample adults in each state, weighting the demographics of the full adult sample (for characteristics such as gender, age, race and education) to match U.S. Census estimates for the full population. They then select "likely voters" based on questions answered by the respondents, without making any further adjustments to the sample's demographics or partisanship.

There are pollsters that weight the subset of "likely voters" by party or to match very specific assumptions about the demographics of those they expect to vote. However, such practices are generally shunned by the national media surveys whose recent results have drawn most of the "skewed poll" criticism."

So if a pollster does indeed end up with a D+13 sample, it won't be because they've made assumptions about how likely members of each party will vote, nor will it be because they've made assumptions about different age, gender and racial groups will vote. It'll be because it happens that more Democrats have answered the likely voter questions in such a way as to make it through the likely voter screen.

[Torie]

Yes, that is the claim, but as I said, just re-weighting based on census data is not a magic wand of accuracy.  Moreover, that still leaves the problem of assuming that those who have land lines, and answer the phone, will have the same voting habits as those who do not within their subgroup. And suppose that 85% if Hispanics say they are likely to vote, to take an extreme example. Does one just throw historical voting propensity habits into the dust bin?  So many conundrums, so little time.

In short, there is no true random sample of those who will be actually voting. It is impossible to obtain such a sample. So you have to try to replicate it, in a zero gravity environment laboratory as it were, and there is no such laboratory so pristine and pure.

[Likely Voter]

The issues with landlines and cell phones is something that polling companies are dealing with. However, it is not something new and certainly would not explain the shift in polling in the last month.

[Torie]

The issues with landlines and cell phones is something that polling companies are dealing with. However, it is not something new and certainly would not explain the shift in polling in the last month.

Yes, the trend thing is harder to blow off as  a posited ephemeral zephyr against the prevailing wind direction, unless the internal methodology changed. That is why I always find more interesting intra-polling firm trends, than I do inter-polling ones.

[Averroës Nix]

I found Nate Silver's take on this question interesting. Here's what he found.



His headline, "Poll Averages Have No History of Consistent Partisan Bias," rings true, but the most important lesson that I see in these numbers is that polling is usually somewhat accurate, but occasionally wildly off (1980, 1996).

Another table shows that state-level polling has been more accurate, particularly in recent years. That's not good news for Romney! But I came away less confident in Obama's lead (which I expect to decline between now and election day, anyway) - although I still believe that the preponderance of evidence suggests that since the Dem convention, he has become more than just a slight favorite.

[ajb]

Yes, that is the claim, but as I said, just re-weighting based on census data is not a magic wand of accuracy.  Moreover, that still leaves the problem of assuming that those who have land lines, and answer the phone, will have the same voting habits as those who do not within their subgroup. And suppose that 85% if Hispanics say they are likely to vote, to take an extreme example. Does one just throw historical voting propensity habits into the dust bin?  So many conundrums, so little time.

In short, there is no true random sample of those who will be actually voting. It is impossible to obtain such a sample. So you have to try to replicate it, in a zero gravity environment laboratory as it were, and there is no such laboratory so pristine and pure.

The concerns you raise here are valid ones, obviously. But your previous post had conveyed the misleading sense that Mark Blumenthal had said that reputable polling firms used some kind of model to project what share of the electorate would belong to different demographic groups. I just wanted to make sure we were all clear that reputable polling firms don't do that, and that Mark Blumenthal wasn't being cited as an authority claiming that they do.
There are obviously enough difficulties with polling data as it is without introducing new ones.

[Torie]

Yes, that is the claim, but as I said, just re-weighting based on census data is not a magic wand of accuracy.  Moreover, that still leaves the problem of assuming that those who have land lines, and answer the phone, will have the same voting habits as those who do not within their subgroup. And suppose that 85% if Hispanics say they are likely to vote, to take an extreme example. Does one just throw historical voting propensity habits into the dust bin?  So many conundrums, so little time.

In short, there is no true random sample of those who will be actually voting. It is impossible to obtain such a sample. So you have to try to replicate it, in a zero gravity environment laboratory as it were, and there is no such laboratory so pristine and pure.

The concerns you raise here are valid ones, obviously. But your previous post had conveyed the misleading sense that Mark Blumenthal had said that reputable polling firms used some kind of model to project what share of the electorate would belong to different demographic groups. I just wanted to make sure we were all clear that reputable polling firms don't do that, and that Mark Blumenthal wasn't being cited as an authority claiming that they do.
There are obviously enough difficulties with polling data as it is without introducing new ones.

You must be logged in to read this quote.

I put my comment in bold just to do my part in making sure your little point is not lost. I am not sure I quite believe Blumey that all the pollsters other than bad boy Rass actually are census junkies however. I suspect some have their own spices in the cupboard. But that is his claim.

[Small Business Owner of Any Repute]

All other evidence aside, one would think that the pollsters will get better at their craft as time goes on, not worse.

[Torie]

All other evidence aside, one would think that the pollsters will get better at their craft as time goes on, not worse.

Blame cell phones and more persons of color - particularly Hispanics.

[King]

All other evidence aside, one would think that the pollsters will get better at their craft as time goes on, not worse.

Blame cell phones and more persons of color - particularly Hispanics.

Nearly everybody has cell phones now.  It shouldn't effect the results.  Cell phone users are so diverse now that the results of those without them probably equal the results of those with them.

Just like the results of the people who do turn out to vote likely would have been the same if turnout were 100%.

[Politico]

All other evidence aside, one would think that the pollsters will get better at their craft as time goes on, not worse.

Blame cell phones and more persons of color - particularly Hispanics.

Nearly everybody has cell phones now.

Yes, but people who only have cell phones are incredibly difficult to poll relative to people with landlines. That is the point. It's a logistical thing that makes random sampling more difficult than ever before, and many pollsters largely have to assume that these cohorts display identical voting patterns to people with landlines, which is questionable...

[milhouse24]

I would not poll anyone under the age of 30 or 25.
I would not poll anyone who has never donated to a political party or candidate. 

I just think that there are too many "unreliable voters" that "aren't going to vote" 

I know some pollsters ask about "enthusiasm" but that is really vague.  Voters need to back up their answers with money and donations.  Actions prove how reliable someone is to voting.  There are too many voters that will just sit at home rather than go vote. 

I think Obama has the "media saturation" edge now, and he's just better known and like-able.  A lot of fickle voters have not heard of Romney or know enough about him to vote for him. 

[Likely Voter]

Regarding cell phones. May pollsters are now including them in their samples. And Nate Silver pointed out that polls that dont include cell phones have a Romney lean.
http://fivethirtyeight.blogs.nytimes.com/2012/09/19/obamas-lead-looks-stronger-in-polls-that-include-cellphones/

So the issue of polls not including cellphones again cannot explain the Obama trend seen in September. It is a red herring.