Non-Gallup/Rasmussen tracking polls thread

[J. J.]

Um, that's not true.  You can still calculate a Margin of Error for a (relatively) small sample size, and apply it.  The MoE is higher.  It isn't high enough to make that result not highly suspicious.

MOE = 150%

MoE = +/-10%

That subsample = Obviously flawed well out of MoE range

Actually, there were others with similar results.  I raised the question about it.  It seems to be a very small sampling.

Which makes it immune to the laws of margin of error? 

No, just using a really bad sample for that age group.

[Alcon]

Um, that's not true.  You can still calculate a Margin of Error for a (relatively) small sample size, and apply it.  The MoE is higher.  It isn't high enough to make that result not highly suspicious.

MOE = 150%

MoE = +/-10%

That subsample = Obviously flawed well out of MoE range

Actually, there were others with similar results.  I raised the question about it.  It seems to be a very small sampling.

Which makes it immune to the laws of margin of error? 

No, just using a really bad sample for that age group.

Right...which shows that there was probably a deep methodological flaw in this poll, or its sample, or something.  It's not "too small of a sample to be able to generalize" -- the generalization is that the chance that would occur, without their being a sampling issue, is infinitesimal.  Add that into the fact that the poll seems like an outlier in the top-line, and I'm beginning to smell the strong scent of fish.

[J. J.]

Um, that's not true.  You can still calculate a Margin of Error for a (relatively) small sample size, and apply it.  The MoE is higher.  It isn't high enough to make that result not highly suspicious.

MOE = 150%

MoE = +/-10%

That subsample = Obviously flawed well out of MoE range

Actually, there were others with similar results.  I raised the question about it.  It seems to be a very small sampling.

Which makes it immune to the laws of margin of error? 

No, just using a really bad sample for that age group.

Right...which shows that there was probably a deep methodological flaw in this poll, or its sample, or something.  It's not "too small of a sample to be able to generalize" -- the generalization is that the chance that would occur, without their being a sampling issue, is infinitesimal.  Add that into the fact that the poll seems like an outlier in the top-line, and I'm beginning to smell the strong scent of fish.

Well, I'm look looking at this subsample, which is off.  As a full sample, until today, it was in line with the other major polls, even more pro-Obama than BG.

I'm give TIPP a bit more credence because of its track record than I am something like Zogby or Hotline.

[André]

ABC / Washington Post

likely voters:

Obama 54 % (nc)
McCain 43 % (nc)

registered voters:

Obama 54 % (+1)
McCain 41 % (nc)

[Alcon]

Well, I'm look looking at this subsample, which is off.  As a full sample, until today, it was in line with the other major polls, even more pro-Obama than BG.

OK, so you think it's a good poll, and it's just a one-in-several-million error?  Interesting.

[J. J.]

Well, I'm look looking at this subsample, which is off.  As a full sample, until today, it was in line with the other major polls, even more pro-Obama than BG.

OK, so you think it's a good poll, and it's just a one-in-several-million error?  Interesting.

I think it's a good poll, with a bad subsample.

[Lunar]

To be honest, the error is more likely than the raw calculation.  Why?  Because the error wasn't predetermined, we were looking at ALL of their data, which means we could find various age-sample outliers, various race outliers, etc.  We just happened to grasp on the one that is off.

That plus the small sample-size are ONLY two reasons to discard subsamples.  

Discarding subsamples is not a rule and it's not universal, J.J., it's just a rule of thumb.  There's a line somewhere (we could calculate it, based on the number of crosstabs available and known sample sizes, but it'd be way too much work) where the poll crosses over into the unacceptable line and becomes DISQUALIFIED.  If some poll of Iowa shows McCain winning 51% of AA voters, that's not that bad.   If some poll shows McCain within a couple points or whatever on youth voters, that's not that bad.  Once it crosses that line, which is fuzzy at this point, but can be finely calculated, the poll becomes ridiculous.  If some poll showed McCain winning 100% of AA's in Georgia, that's ridiculous.  And this poll, which shows McCain winning youth voters by a 50% margin, becomes disqualified from legitimacy in a simpler manner.

Especially because the topline result is also an outlier.

I don't see how you can logically argue otherwise.

[J. J.]

To be honest, the error is more likely than the raw calculation.  Why?  Because the error wasn't predetermined, we were looking at ALL of their data, which means we could find various age-sample outliers, various race outliers, etc.  We just happened to grasp on the one that is off.

That plus the small sample-size are ONLY two reasons to discard subsamples.  

Discarding subsamples is not a rule and it's not universal, J.J., it's just a rule of thumb.  There's a line somewhere (we could calculate it, based on the number of crosstabs available and known sample sizes, but it'd be way too much work) where the poll crosses over into the unacceptable line and becomes DISQUALIFIED.  If some poll of Iowa shows McCain winning 51% of AA voters, that's not that bad.   If some poll shows McCain within a couple points or whatever on youth voters, that's not that bad.  Once it crosses that line, which is fuzzy at this point, but can be finely calculated, the poll becomes ridiculous.  If some poll showed McCain winning 100% of AA's in Georgia, that's ridiculous.  And this poll, which shows McCain winning youth voters by a 50% margin, becomes disqualified from legitimacy in a simpler manner.

Especially because the topline result is also an outlier.

I don't see how you can logically argue otherwise.

Sorry, but I don't see the importance of a bad subsample, especially since it was repeated during samples that matched other polls.

[Alcon]

Sorry, but I don't see the importance of a bad subsample, especially since it was repeated during samples that matched other polls.

Because it demonstrates that either:

1. There's a methodological error; or,

2. A one-in-many-million chance event has occurred.

Which seems more likely to you, especially when the poll soon thereafter shows a result that seems like an outlier?

[J. J.]

Sorry, but I don't see the importance of a bad subsample, especially since it was repeated during samples that matched other polls.

Because it demonstrates that either:

1. There's a methodological error; or,

2. A one-in-many-million chance event has occurred.

Which seems more likely to you, especially when the poll soon thereafter shows a result that seems like an outlier?

It hasn't been showing the results as being an outlier, at least until today.  It was in line with Gallup and Rasmussen. 

In seems likely  subsample is clearly an outlier, but the whole poll.

[Alcon]

Sorry, but I don't see the importance of a bad subsample, especially since it was repeated during samples that matched other polls.

Because it demonstrates that either:

1. There's a methodological error; or,

2. A one-in-many-million chance event has occurred.

Which seems more likely to you, especially when the poll soon thereafter shows a result that seems like an outlier?

It hasn't been showing the results as being an outlier, at least until today.  It was in line with Gallup and Rasmussen. 

In seems likely  subsample is clearly an outlier, but the whole poll.

So, you're maintaining that the subsample was just a one-in-several-million event? Yes or no?

[Lunar]

If a pollster is more likely to be hit by a meteor and killed than get the result they release, then their model is bad.

[J. J.]

Sorry, but I don't see the importance of a bad subsample, especially since it was repeated during samples that matched other polls.

Because it demonstrates that either:

1. There's a methodological error; or,

2. A one-in-many-million chance event has occurred.

Which seems more likely to you, especially when the poll soon thereafter shows a result that seems like an outlier?

It hasn't been showing the results as being an outlier, at least until today.  It was in line with Gallup and Rasmussen. 

In seems likely  subsample is clearly an outlier, but the whole poll.

So, you're maintaining that the subsample was just a one-in-several-million event? Yes or no?

I'm saying the subsample is bad, but a bad subsample doesn't invalidate the poll.

[Alcon]

I'm saying the subsample is bad, but a bad subsample doesn't invalidate the poll.

So, in other words, you are maintaining that it's a one-in-several-million event, and not a flaw in the methodology.  Why?

[Moooooo]

What type of lead among this age group are other pollsters showing?

Could this skew the top line #'s?

[Alcon]

What type of lead among this age group are other pollsters showing?

All, that I know of, are showing at least 15-20 points.

Could this skew the top line #'s?

To some (small) degree, but I think the bigger point is that it points to a likely flaw in methodology.  That kind of result doesn't just materialize out of statistical noise.

[Verily]

What type of lead among this age group are other pollsters showing?

All, that I know of, are showing at least 15-20 points.

Could this skew the top line #'s?

To some (small) degree, but I think the bigger point is that it points to a likely flaw in methodology.  That kind of result doesn't just materialize out of statistical noise.

A 70-point swing (from a 50-point McCain lead to a 20-point Obama lead) in 10% of the sample is a 7-point swing in the overall result, to an Obama lead of just over 8 points. Or, in other words, the difference between IBD/TIPP being one of Obama's worst polls overall and his best tracking poll and above his RCP average lead.

That might overstating the case a bit if you want to say that the youth vote is the 9.3% it was in 2004 (it won't be, and will probably be more like 12%) and if you want to be conservative in your estimate of Obama's lead among 18-25s (say, 12-15 points instead of 20), but the skew among youth is hurting Obama's lead by at least five, maybe as many as eight or nine points.

It's not at all irrelevant to the topline numbers even ignoring the obvious problems that must be inherent in the methodology if they're turning up such awful samples.

[Rowan]

What type of lead among this age group are other pollsters showing?

All, that I know of, are showing at least 15-20 points.

Could this skew the top line #'s?

To some (small) degree, but I think the bigger point is that it points to a likely flaw in methodology.  That kind of result doesn't just materialize out of statistical noise.

A 70-point swing (from a 50-point McCain lead to a 20-point Obama lead) in 10% of the sample is a 7-point swing in the overall result, to an Obama lead of just over 8 points. Or, in other words, the difference between IBD/TIPP being one of Obama's worst polls overall and his best tracking poll and above his RCP average lead.

That might overstating the case a bit if you want to say that the youth vote is the 9.3% it was in 2004 (it won't be, and will probably be more like 12%) and if you want to be conservative in your estimate of Obama's lead among 18-25s (say, 12-15 points instead of 20), but the skew among youth is hurting Obama's lead by at least five, maybe as many as eight or nine points.

It's not at all irrelevant to the topline numbers even ignoring the obvious problems that must be inherent in the methodology if they're turning up such awful samples.

Let's examine every internal then to see where there could be flaws to add points to your candidate.

[Alcon]

I want to see the internals first.

Let's examine every internal then to see where there could be flaws to add points to your candidate.

Time elapsed:  2 minutes, 51 seconds.

[Rowan]

I want to see the internals first.

Let's examine every internal then to see where there could be flaws to add points to your candidate.

Time elapsed:  2 minutes, 51 seconds.

Sorry, I have John Kerry disease.

[Verily]

What type of lead among this age group are other pollsters showing?

All, that I know of, are showing at least 15-20 points.

Could this skew the top line #'s?

To some (small) degree, but I think the bigger point is that it points to a likely flaw in methodology.  That kind of result doesn't just materialize out of statistical noise.

A 70-point swing (from a 50-point McCain lead to a 20-point Obama lead) in 10% of the sample is a 7-point swing in the overall result, to an Obama lead of just over 8 points. Or, in other words, the difference between IBD/TIPP being one of Obama's worst polls overall and his best tracking poll and above his RCP average lead.

That might overstating the case a bit if you want to say that the youth vote is the 9.3% it was in 2004 (it won't be, and will probably be more like 12%) and if you want to be conservative in your estimate of Obama's lead among 18-25s (say, 12-15 points instead of 20), but the skew among youth is hurting Obama's lead by at least five, maybe as many as eight or nine points.

It's not at all irrelevant to the topline numbers even ignoring the obvious problems that must be inherent in the methodology if they're turning up such awful samples.

Let's examine every internal then to see where there could be flaws to add points to your candidate.

In most cases, internals do not make a big difference. Or, rather, the skew from internals is very small, say 0.2 points one way and then 0.3 points the other way and then 0.1 points back the first way from three skews in three subsamples in one poll. This poll has a single subsample which is so absurdly far off that it massively screws up the overall result as well.

But you can accuse me of being a hack all day long. It just makes you cuter.

[J. J.]

I'm saying the subsample is bad, but a bad subsample doesn't invalidate the poll.

So, in other words, you are maintaining that it's a one-in-several-million event, and not a flaw in the methodology.  Why?

No, I'm saying that the poll isn't primary designed to accurately measure a subsample.

[Alcon]

I'm saying the subsample is bad, but a bad subsample doesn't invalidate the poll.

So, in other words, you are maintaining that it's a one-in-several-million event, and not a flaw in the methodology.  Why?

No, I'm saying that the poll isn't primary designed to accurately measure a subsample.

It has to be either a one-in-a-million event or a flaw in the methodology.  It can't be neither, lol.

[Lunar]

I'm saying the subsample is bad, but a bad subsample doesn't invalidate the poll.

So, in other words, you are maintaining that it's a one-in-several-million event, and not a flaw in the methodology.  Why?

No, I'm saying that the poll isn't primary designed to accurately measure a subsample.

Yes, but at some level you have to question the methodologies, especially if the topline result isn't in line with other polls.

The subsample can reveal the methodological weaknesses of a poll.  If a subsample shows 500% growth among elderly voters, then perhaps the poll is doing something funky.  It's wrong to pick apart moderate subsample weaknesses (say McCain winning 15% of blacks, a common subsample weakness with about the same sample size as Youth), but at some point we got to take a step back and say that this poll is probably doing something wrong.  The odds are simply one in a trillion that this poll doesn't have an overarching flaw.

At what point do you, J.J., question a sub sample?  Is a one and a gazillion bazillion chance not enough for you?  Does McCain need to win 100% of Blacks in Mississippi?

[J. J.]

I'm saying the subsample is bad, but a bad subsample doesn't invalidate the poll.

So, in other words, you are maintaining that it's a one-in-several-million event, and not a flaw in the methodology.  Why?

No, I'm saying that the poll isn't primary designed to accurately measure a subsample.

It has to be either a one-in-a-million event or a flaw in the methodology.  It can't be neither, lol.

The "flaw in the methodology" of getting the subsample or a bad subsample does not invalidate the whole sample.  The 45-64 sample seems skewed to Obama and I don't believe it; that doesn't invalidate the poll.