Non-Gallup/Rasmussen tracking polls thread

[Alcon]

So have we figured out yet why McCain did a couple of percent better than the polls in general, apparently?  You know what?  We might not really know. Oh the horror, the horror!  By the way, Obama will be close to a 7% lead before this is all over I suspect.

Historically, it's common for the candidate with a large lead to overpoll a small amount. Yes, even the whites ones, JJ.

This is true, and an argument to be had if Obama overpolled to statistical significance.

[Alcon]

No, as a tracker, it's the same as a state poll.  It's still interviews being conducted over a three-day period.  Muhlenberg's three-day sample size was 618, bigger than Insider Advantage, ARG, Zogby, and only 7 fewer responses than Mason-Dixon.  No reason to throw it out.  It's a minor point, though, but unless you know something about Muhlenberg I don't, I have to include it for consistency if we're including all polls.

So, in other words, you want option B?  That's ([Obama minus final Obama]-[McCain versus final McCain]).

And to clarify, you want all polls, not just the recognized national ones I listed?

Once I get your response on that, I'll compile the results and check for statistical significance.

[Alcon]

Actually, that wasn't true in a race I looked at a while back, Casey in 2006.  He underpolled slightly.  Both Swann and Rendell underpolled

It is not an exceptionless rule (what is -- ever?).  However, FiveThirtyEight did a table and determined that it does tend to happen presidentially.

You didn't really answer either of my questions.  I can't start the procedure until you do.

[Alcon]

But trackers are the same as a three-day poll.  Do you understand that?  There is no basis for considering them different entities.  They are the same as a three-day poll.  There's no difference.

Anyway, you didn't answer either of my questions, again.  Do you want all polls, regardless of pollster?  Is option B -- ([Obama minus final Obama]-[McCain versus final McCain]) -- acceptable?

I can even arbitrarily exclude Muhlenberg if you really really want.  Just realize that it makes no sense!

[Alcon]

OK.  While I'm working on this, you can explain to everyone how the Muhlenberg tracking poll (612 phone interviews conducted over three days) is worth excluding while Insider Advantage (588 phone interviews conducted over three days) should be included.

go!

[Alcon]

Too slow!  (But, seriously, keep working on that explanation.)

Anyway, done.  Excluding Muhlenberg and the UNH tracker, which I think is ridiculous and arbitrary but whatever, we have 135 state polls during the specified time period.

On average, they underestimated Obama -- yes, Obama -- by a 1.02% margin.  This was not statistically significant.

Of the 50 states, 37 had polling during this period.  Obama was under-estimated in 15, and over-estimated in 21.  Montana was an exact statistical hit.  More of McCain's 21 were small margin errors (i.e., <2.5%) or based on a single poll -- hence why all other measures showed Obama being the one under-estimated.

None of the aforementioned 36 were statistically significant, save for the following:

* Obama under-estimations in Massachuestts, Michigan, Nevada, New Mexico and Pennsylvania and Vermont.

* Obama over-estimations in Alaska (serious pollster meltdown there), Arizona, Arkansas, Iowa and Wyoming.

So, we have a statistically insignificant overall under-estimation of Obama.  There were five six states where Obama was under-estimated to statistical significance, and four where he was over-estimated.

Now, speaking of individual polls, 26 of 135 (19%) were off to statistical significance, which is actually a little bit better than a perfectly-conducted batch should be (the joys of weighting)

In any case, conclusions:

* On average, state polls slightly under-estimated Obama, but not to statistical significance.

* As one would expect with his overall being slightly under-estimated in the state polls, Obama's under-estimation achieved statistical significance in more states than did McCain's (6 vs. 4).

Iowa Arizona and one Obama-underpoll state (closed the window Kansas) narrowly missed SS.

* While Obama was over-estimated in more states than McCain, they tended to be states with few polls, and thus more potential for noise.  Of Obama's statistically significant under-estimations, all but MA had a good number of polls; only two of McCain's did (AK, AZ IA).

In closing, there is no strong evidence that polls under-estimated Obama.  However, there is no evidence whatsoever of a Bradley Effect, or any kind of Obama over-representation.  If it happened, it was eaten up by noise, an opposing pro-Obama effect, or perhaps was all a dream.

[Alcon]

And just for fun, to further drive the point in, I looked at 538's time/sample size-adjusted model.  This is a bit more friendly to your thesis.  Obama was only under-estimated by 0.13% there, which is a lot further from statistical significance than 1.02%.  The biggest botch here wasn't Alaska, though; it was D.C., where the polls were off a whopping 20 points.  (Hawai'i was second-worse, another Obama under-estimation, then Alaska barely before another Obama under-estimation in Vermont.)

Convinced that you might be wrong about the Bradley Effect, yet?  Even a little?  My guess is "no."

(How's the tracker poll explanation going, btw?)

This is fun

[Alcon]

I think it's time to retroactively throw some polls out.

Amusingly, throwing tracking polls out (very barely) increases Obama's under-estimation, since Muhlenberg was a little bit less pro-McCain in PA than UNH was pro-Obama in NH.

[Alcon]

Still waiting to find out why a three-day tracker is different than a three-day poll, btw.

[Alcon]

Still waiting to find out why a three-day tracker is different than a three-day poll, btw.

...wait, you mean that polling accuracy fluctuates up and down, possibly due to poor weighting, and it's stupid to apply analysis to tiny amounts of error without statistical significance, especially if it doesn't stand out from the overall sample?

Take your radical ideas back to Europe, scum

[Alcon]

OK.  While I'm working on this, you can explain to everyone how the Muhlenberg tracking poll (612 phone interviews conducted over three days) is worth excluding while Insider Advantage (588 phone interviews conducted over three days) should be included.

go!

I think it's a different purpose of the poll.  Tracing polls are, well, intended to track.  A state poll is intended to be a snapshot of the electorate.

A three-day tracking poll is not different from a three-day state poll.  Both are a poll conducted over three days.  Their methodology is totally identical.  They are both a "snapshot" of three days.  There are actually non-tracking polls conducted over more than three days.

In PA, SurveyUSA, Zogby and Strategic Vision all fielded polls that are less of a "snapshot" than the Muhlenburg tracker.

[Alcon]

I've given you my reason.  They basically have a different purpose. 

I pointed out how that reason is totally invalid.  A three-day tracking poll is conducted the exact same way as a three-day poll.  You could not distinguish the two.  Everything about them is the same.  Unless you can explain how the specific methodology is different, or demonstrate that it is, you're just making stuff up.

Also, part of the same sample is included in another tracking poll. 

Huh?

[Alcon]

Yeah, but a tracking poll released on November 1st is the same thing as a poll conducted from October 29th to 31st.  So, since we're including polls from 10/28 otherwise, why throw a poll from the 29th-31st out?

[Alcon]

J. J.,

Had someone clarify what you meant.

I posted a Spreadsheet with the poll data.  If you want to compare this to finals, to see where Obama and McCain under- and over-performed, be my guest.  It's useless information, though; it relates more to how hard undecideds are pushed, and how third-parties are managed.  Either way, the table I presented is the closest useful version of that.  I mean, really, what does both McCain and Obama under-polling say?  Double Bradley Effect?  Really:  Probably university pollsters, Mason-Dixon, and other non-pushy pollsters.

And I see your tracker point, but I'm only including the last tracker as if it were a three-day poll.  I'm not including any previous trackers, as you're right, that would be double-counting samples.  Theoretically, I could include multiple three-day periods, as long as they don't overlap.  But in the above, I included no trackers, per your request.  But I specifically said I'm including the last day only, so the problem you raise doesn't come up.

[Alcon]

You need to read my second post, it responds to both things you just said.

[Alcon]

I don't have spread sheet.  However, the purpose to look at where both candidates underperformed or overperformed.  This is more to rule out true undecides, really bad polling.  If Obama had 48% and McCain had 46% in a state poll, and the result was 52% to 48%, both candidates underpolled.  That has to be taken into account.

I can do that, but before I do, I want to know what methodology we're going to use to make conclusions from it.  I don't want to totally waste my time with what I suspect will amount to "where did low-push pollsters poll?"

I could deal with that, so long as there wasn't any double counting.  That was my concern. 

Good stuff.  Muhlenberg takes Obama's under-polling in PA up a notch, and down a notch with UNH in NH.  It does not change the overall rates or any statistical significance.

[Alcon]

What I'm looking at is a pattern.  If both candidates tended to underpoll in a state, it probably is more a function of the polling or of true undecided voters than anything else..  I've been looking more at cases where McCain underpolled and Obama didn't.  I'd like to knock out those states where both underpolled.

Even though you admit that's more of a function of polling techniques?  Be my guess  The Spreadsheet is there for you.  Tell me how it goes.

Preview: More McCain underpoll states get knocked out, nearly evening them up; statistical significance of the aforementioned states, obviously, is unaffected.  Obama's overall underpoll margin increases slightly.

So, where are we at?  No statistical evidence of a Bradley Effect for every single test you've had me undertake so far.  Beginning to get at all skeptical? 

edit: removed unintentional passive-aggressive language.  sorry.

[Alcon]

Why would one candidate underpolling not also potentially be an indication of polling techniques?  Why not use the method I developed, wherein you subtract one candidate's underpolling from the other?

Trackers are not better than state polls.  They're the same kind of poll, just done nationally.  The number of samples in the state polls greatly outweighs the national trackers.  The national trackers were not wrong to statistical significance.

You've ignored the fact that Obama under-polled overall, in favor of him overpolling in more states -- which I already told you does not meet statistical significance.  I told you which states statistical significance was met in.  Especially considering that winning presidential candidates tend to over-poll, that's meaningful.

And, yes, you are looking for a Bradley Effect.  Desperately.

[Alcon]

One think I noticed in 2006 (3 out 5 times) was that the Black candidate didn't overpoll as much as the white candidate underpolled

ok?

I didn't say trackers were "better."  I said something occurred on trackers (and frankly something I was not expecting).

Yea, but trackers are just one different sort of poll.

I would say 3 out of 7 outside the MOE was significant.

Which were those?  What were the MoEs, and ho  did the candidates track?

I'm not ignoring anything, in fact, I'm asking a question about it.  I would expect that on some polls, both would underpoll; that could be due to undecides.  I'm interested in cases where both candidates did not underpoll.

Remember, I thought it would be very weak, but present.

So, you're comfortable with using data that doesn't reach statistical significance to make a determination which could have alternative explanations.

[Alcon]

J. J., sincerely, here's why people don't especially take you seriously as an analyst.

3/5 races with black candidates in one year = You bring up multiple times

The 538 thing Lunar mentioned (and I have, several times) = You've totally ignored

Coincidentally, #1 justifies your theory and #2 rejects it.  #2 is, by any reasonable mathematical and scientific measure, more significant.  And yet, which one gets the air time?

Just a thought.

[Alcon]

I provided you with the data.  You need to download a spreadsheet program and parse it for yourself.  I don't think your test is meaningful; I think it relies more on pollster methodology differences.  I already performed one test you asked me to do.  It came back with data that disputed your thesis, which you seem to have summarily ignored.  But I already spent an hour doing a meaningful test; you need to spend fifteen minutes getting the software, and then doing your own test.  Especially considering I dispute the meaningfulness of this test, and a test whose meaning we agreed on you've outright ignored.

You never see the data?  I posted a table of data and the full source spreadsheet.  How is that not seeing the data?  Because you don't have spreadsheet software?  That's not my responsibility.  In fact, I've never seen you do a single mathematical test.  Where's your science? 

(Reminder: Checking three races from 2006 and not even bothering to do statistical significance checks = not scientific!  The information you ignored = scientific!)

[Alcon]

I'm trying to provide you with data.  I've linked to it, but you say your computer is not able to open it.  Why are you accusing me of not providing data, and mocking me, because of your technical issues?  You haven't even told me what format you can open.

That's not a rhetorical question, by the way.  Why are you mocking me because of your own technical issues?  I want that answered before I do any further data processing for you.

(P.S. I'm an agnostic.  calling 'science' my god is just stupid, not offensive to me.  it doesn't even really make sense as an insult.)

[Alcon]

So, J. J., are you going to explain why you were mocking me because you had computer issues?  And claiming that I refused to provide you the data, when in fact you just did not know how to open it?

[Alcon]

BRADLEY EFFECT MAP


MAP I MADE BY FLIPPING A COIN


Not sure about map 1, seems kinda random, but I definitely see some major Bradley Effect going on in map 2.  Must be a racist quarter.