Non-Gallup/Rasmussen tracking polls thread

[Alcon]

So, 538's poll average-based model doesn't show the effect you allege because...?

[J. J.]

So, 538's poll average-based model doesn't show the effect you allege because...?

1.  Would you provide a link?

2.  Would you explain why, in terms of national polls, none were outside the MOE, except those showing a lead for Obama?

[Alcon]

1.  Would you provide a link?

fivethirtyeight-dot-com

They weight by sample size, and time elapsed since release, and pollster record.  But unless you can explain why doing so removes the Bradley Effect, your suggestion that Obama under-performed vs. state polls is unfounded.

2.  Would you explain why, in terms of national polls, none were outside the MOE, except those showing a lead for Obama?

n=2?

[J. J.]

1.  Would you provide a link?

fivethirtyeight-dot-com

They weight by sample size, and time elapsed since release, and pollster record.  But unless you can explain why doing so removes the Bradley Effect, your suggestion that Obama under-performed vs. state polls is unfounded.

Please provide a link.  The only thing I've found was this:  http://www.newsweek.com/id/165030/page/2

You must be logged in to read this quote.

n=2?
[/quote]
[/quote]

n=3, if I understand your point.

We've been looking at a number of tracking polls:

Zogby (understandably)
Gallup
ABC/WP
(These overcounted and are outside the MOE)

TIPP
(overcounting, but inside the MOE)

Rasmussen
R2K
(Undercounted, but inside the MOE)

[Alcon]

Sigh.

fivethirtyeight-dot-com = www.fivethirtyeight.com

C'mon man

[J. J.]

Sigh.

fivethirtyeight-dot-com = www.fivethirtyeight.com

C'mon man

The section the particular point you are making, "538's poll average-based model."

The Supertracker?

[Alcon]

Sigh.

fivethirtyeight-dot-com = www.fivethirtyeight.com

C'mon man

The section the particular point you are making, "538's poll average-based model."

The Supertracker?

"Trend-Adjusted" under the right-hand side.  Did you bother to read the FAQ, or look through the entire page?  It's pretty self-explanatory.

Yes, that method is used with the Supertracker.

[J. J.]

Sigh.

fivethirtyeight-dot-com = www.fivethirtyeight.com

C'mon man

The section the particular point you are making, "538's poll average-based model."

The Supertracker?

"Trend-Adjusted" under the right-hand side.  Did you bother to read the FAQ, or look through the entire page?  It's pretty self-explanatory.

Yes, that method is used with the Supertracker.

Well, one difference is that I look at only the last week of polling, in the case of tracking, final polls.  I treat polls as a snapshot of the electorate and I want the photo to be close to the event.  So that would be one difference, probably the main one.

A also look for trends, basically where McCain underperformed and Obama didn't under preform, or where Obama was in the MOE and McCain wasn't.

[Alcon]

Wait, so the Bradley Effect was contingent upon the place in the election cycle?

Besides, the weighting was done in such a way that polls in the last week were hugely over-weighted relative to earlier ones.

[Gustaf]

We should remember in the midst of glorifying 538 that all his talk about cell phones and young voters emerging turned out to be bull.

But, JJ, there clearly wasn't a Bradley effect to be noted this time around. That is pretty obvious. If anything, Obama did slightly better than the polls predicted but they were generally just pretty much spot on.

[J. J.]

Wait, so the Bradley Effect was contingent upon the place in the election cycle?

Besides, the weighting was done in such a way that polls in the last week were hugely over-weighted relative to earlier ones.

Alcon, what I am looking at is basically the number of voters who don't answer the polls accurately.  In many cases, they answer undecided.  There were also true undecideds out there, but they tend to make up there minds before election day.  For this reason, I look at ending polls and really don't want to to include polls that are 30 days old, or 15 days old, or 10 days old.

There were also legitimate shifts in opinion within the last 30 days.  PA is a good example.  The polls closed about a week out, then bounced back.

No weighting just the last snapshot.

Gustav,  in the national tracking, there was.  We had three national tracking polls (out of six) that over counted Obama and did so outside outside of the MOE.  Two of them were not Zogby.    I suggested that there would be a small Bradley Effect, 1-2 points.  That small effect seems to be there.  Now, did it make a difference?  No.  Was it a polling artifact?  Yes.  A large one? No.  An effect that has been measured over time and is declining?  Yes.

If I had to say what it was, at this point, 1-3 points, in this election.

[Gustaf]

Wait, so the Bradley Effect was contingent upon the place in the election cycle?

Besides, the weighting was done in such a way that polls in the last week were hugely over-weighted relative to earlier ones.

Alcon, what I am looking at is basically the number of voters who don't answer the polls accurately.  In many cases, they answer undecided.  There were also true undecideds out there, but they tend to make up there minds before election day.  For this reason, I look at ending polls and really don't want to to include polls that are 30 days old, or 15 days old, or 10 days old.

There were also legitimate shifts in opinion within the last 30 days.  PA is a good example.  The polls closed about a week out, then bounced back.

No weighting just the last snapshot.

Gustav,  in the national tracking, there was.  We had three national tracking polls (out of six) that over counted Obama and did so outside outside of the MOE.  Two of them were not Zogby.    I suggested that there would be a small Bradley Effect, 1-2 points.  That small effect seems to be there.  Now, did it make a difference?  No.  Was it a polling artifact?  Yes.  A large one? No.  An effect that has been measured over time and is declining?  Yes.

If I had to say what it was, at this point, 1-3 points, in this election.

There are plenty of reasons why the polls were wrong. Gallup obviously just screwed up. There isn't as of yet any obvious reason as to why the Bradley effect would show up in some trackers and not in others.

The relevant fact is to look at the state level, imo and possibly at subsets like white blue collar workers. If there was a Bradley effect it should have been evident in states like PA, OH or NC imo. And it clearly wasn't. If anything, Obama did a tiny bit better in those states than the polls predicted.

I'm not saying there weren't one or two Bradley voters somewhere but it was completely insignificant.

[Nym90]

Obama did end up doing slightly better in the national polls than he did on election day. It could well have just been the MOE, or maybe the undecideds broke for McCain for millions of other reasons other than race, or perhaps Democratic enthusiasm, as high as it was, was still overestimated.

No evidence at all though that the state polls overall were biased to one candidate or the other.

[Rowan]

Why are you guys still arguing with him? It was clearly the Bradley Effect.

[Alcon]

We should remember in the midst of glorifying 538 that all his talk about cell phones and young voters emerging turned out to be bull.

True, but not related to the model.

In any case, I guess we're just not understanding J. J. on the following:

1. Weighting based on pollster quality, age and poll conduct time somehow gets rid of the Bradley Effect, in a way I assume J. J. will refuse to explain.

2. Any over-performance by Obama is not the Bradley Effect.  Any under-performance probably is, despite the fact that Kerry under-performed polls, too.  Was his West Virginia performance the Bradley Effect?

3. Instead of using fancy-schmancy state polls, we should stick to three trackers, two of which "are not Zogby."

4. The Bradley Effect not showing up in states with racial tension isn't a sign of anything, for some reason.  Even if a state has no real history of racial strife issues (Utah), it should be assumed to be the Bradley Effect.

5. J. J. was going to argue for this, if he could find any remotely feasible opportunity to, no matter how intellectually dishonest it is.

[J. J.]

We should remember in the midst of glorifying 538 that all his talk about cell phones and young voters emerging turned out to be bull.

True, but not related to the model.

In any case, I guess we're just not understanding J. J. on the following:

1. Weighting based on pollster quality, age and poll conduct time somehow gets rid of the Bradley Effect, in a way I assume J. J. will refuse to explain.

No, I'm suggesting that looking at two week old polls isn't that good.  I have explained the snapshot effect.  Going back, even with reducing the weighting, of polls 2-4 weeks ago, might show trending, but not the snapshot.

You must be logged in to read this quote.

So far, we've had one state where Obama overperformed and more than a half dozen McCain overperformed.

You must be logged in to read this quote.

No, I've been looking both national tracking polls and state polls.

You must be logged in to read this quote.

What makes you think that it's a "racial tension" issue.  You are assuming something about the Bradley Effect that has never been claimed.

You must be logged in to read this quote.

I didn't have up the national polls.  It is intellectually dishonest to make claims about "racial tensions" or claims that I'm not looking at state polls, when I've been discussing them.  :rolleyes:

It certainly is intellectually honest when I said, prior to the election, I thought PA could exhibit a Bradley Effect, and now say that the data doesn't show it.

[Alcon]

Didn't you suggest that PA is likely to have the Bradley Effect because of racial tension?  (Caveat:  I was wrong to say that Utah has never had racial issues)

Okey doke, give me a list of pollsters.  Give me a timeframe.  We will agree on a methodology.  Either pick margin-vs.-final (margin minus final margin) or relative candidate-showings-vs.-final ([Obama minus final Obama]-[McCain versus final McCain]).  Then, I will run a poll compilation and test for statistical significance.

If there is no statistical significance, will you finally admit you are wrong -- or at least overestimated the Bradley Effect to the extent it was eaten up by noise/something else?

If there is statistical significance, will you admit that there are plausible reasons other than a Bradley Effect?  At that point, we could debate the relative merits of the possible explanations--but only if there is statistical significance.  Otherwise, seriously, I could claim there's any damn effect and just believe in it no matter what.  Useless.

So, agreed?  Again, you get to outline the specific methodology.  Hell, we can do both, and with multiple timeframes.  It's your choose.

[J. J.]

Didn't you suggest that PA is likely to have the Bradley Effect because of racial tension?  (Caveat:  I was wrong to say that Utah has never had racial issues)

No, the only thing I suggested was that it might be cultural.  Phil, Ice Hockey, and a slew of others (Murtha) basically said the same things.  It was more of "they're rednecks."

You must be logged in to read this quote.

Okay.  All polls from 10/28.  We can argue if the poll is complete crap or not later. 

I'm looking at:

1.  Polls where McCain underpolled and Obama didn't underpoll.  For example:

Poll A:  Obama 48, McCain 44.  Actual result:  Obama 52, McCain 46.  This result would not show a true underpolling.

Poll B:  Obama 52, McCain 40.  Actual result:  Obama 52, McCain 48.  This result would show a true underpolling.

2.  Polls where McCain polled below MOE.

We'll start with that.

I'm looking for a polling artifact.  Except in really close races, it shouldn't make a difference in the result.

I really thought that we would not see it in national polls, but we did.  It didn't occur in some states where I thought it would occur, either.

[Alcon]

No, you can tell me which polls are "complete crap" now.  You don't do an experiment and then toss out data points afterwards.  Why would you?  That serves no purpose whatsoever other than to potentially introduce bias.

[Lunar]

No, you can tell me which polls are "complete crap" now.  You don't do an experiment and then toss out data points afterwards.  Why would you?  That serves no purpose whatsoever other than to potentially introduce bias.

I'm not reading the other posts, but there should be a standard for what polls are complete crap too, right?  538 doesn't toss out any polls, it just weights them on historical [primaries] accuracy...

I'll read the rest when I've cooled down in a week or two.

[J. J.]

No, you can tell me which polls are "complete crap" now.  You don't do an experiment and then toss out data points afterwards.  Why would you?  That serves no purpose whatsoever other than to potentially introduce bias.

Alcon, I'm saying that I'm willing to look at it in the opposite direction.  You can have a poll that is terrible, but can still show an accurate result.  I'm will to say, even though I don't really think that poll is not particularly good, it got it right.  It's only going to be when that poll shows an undercounting for McCain, that I might we willing to say, "Well this might be a polling problem."

Let me put it this way.  In PA, we have Marist, which wouldn't be counted because it was out 10/27, but let's say it came out on 10/28.  It didn't undercount Obama, but did McCain.  It fits my criteria otherwise.  You could look at that poll and say, "Here is an example."  I'd be will to look at that and say, "That is not a particularly good poll." 

In other words, if the effect seems to be present, I'd say, "Let's go back and look at the specific polls."

If this shows up, I'd also like to look at margins, but let's look at this first.

[Alcon]

That's fine, J. J., but I'm not going to let you throw out polls after I do the analysis, lol.  That would be incredibly unscientific.  My argument isn't with the concept that "some polls are crappy," it's with ex post facto toss-outs.

Are you willing to list which polls you consider acceptable so as to look at this objectively, or not?

[J. J.]

That's fine, J. J., but I'm not going to let you throw out polls after I do the analysis, lol.  That would be incredibly unscientific.  My argument isn't with the concept that "some polls are crappy," it's with ex post facto toss-outs.

Are you willing to list which polls you consider acceptable so as to look at this objectively, or not?

No, I'd to look at every state and research every poll.  I want to look to see which states seem to have a B.E. and then see if some of those polls can be factored out.

If we were looking at the seven national tracking polls, for example, we'd get four polls that undercounted McCain, three that had it out of the MOE, the that undercounted McCain and none that undercounted him outside of the MOE.

We might discuss matters and say that we shouldn't count Zogby and Y2Kos (possibly due to the weighting, and I still don''t get how it undercounted Obama).   We'd then have 2 outside of the MOE, one inside underrcount, and two overcounts.

[Alcon]

Well, you said you want to look at state and national polls.  State polls have many more interviews, and are many greater in number, than national polls.  Thus, can we agree that they're the best place to find the Bradley Effect?  There's no logical reason to weight national polls in your determination any more than state polls.  They're the same thing, but with a national sample, and there's fewer of them.

Now:  Why would you throw pollsters out after testing for the Bradley Effect?  Seems that would just potentially introduce unconscious personal bias.  No reason not to throw them out now.  Let's do that, and then I'll test for you.

So, choose which of the following pollsters you want to throw out.

ARG
CNN
Field
Insider Advantage
Los Angeles Times
Marist
Mason-Dixon
National Journal
PPP
Quinnipiac
Rasmussen
Research 2000 (DailyKos)
Selzer
Strategic Vision
SurveyUSA
YouGov
Zogby (phone)

Now, decide what time period you want polls from, and which of the two methods I mentioned you want to use.

We're applying an objective mathematical test to figure this one out.  Then, we can see whether there's mathematical support for a Bradley Effect or not.  Then, we find out the truth, right?

[J. J.]

Alcon, look at the last week of polling.  That would be step one.

Looking at the polls afterward will probably be favorable to the proposition that there is no Bradley Effect.  Now, if you don't want to do it, fine, but I don't want here, "Oh, you have to look at these bad polls," afterward.

On this thread, I've noted that the national tracks show something with Zogby, and posters get annoyed because I'm including Zogby, then I remove Zogby, and I still hear, "Well what about these other polls."