Gallup Tracking Poll Thread [Obama vs McCain]

[Alcon]

Or they both do and will soon be regressing to the mean.

Actually, the odds are 1 in 400 that this will happen.

Based on what?

[Eraserhead]

Or they both do and will soon be regressing to the mean.

Actually, the odds are 1 in 400 that this will happen.

Based on what?

J.J.'s 267th rule of elections.

[Lunar]

Or they both do and will soon be regressing to the mean.

Actually, the odds are 1 in 400 that this will happen.

WTF?  Why can't reality be in between the two polls?  Isn't that the most likely scenario?

[I spent the winter writing songs about getting better]

J. J. would probably fail a 7th grade math test. Remember how Wisconsin was tightening near the end?

[Iosif]

J. J. would probably fail a 7th grade math test. Remember how Wisconsin was tightening near the end?

What are you talking about? He's in Mensa!

[J. J.]

Or they both do and will soon be regressing to the mean.

Actually, the odds are 1 in 400 that this will happen.

Based on what?

1 in 20 polls is an outlier.  For each poll to be an outlier, that's 1 x 1 to 20 x 20 to 1 to 400.

One appears to be an outlier, one isn't.  For both to be an outlier, the odds are 1 to 400.

[Sam Spade]

Personally, I think Rasmussen has an outlier sample in its mix (from yesterday), but whatever.  I also don't think Obama is up by 11.

[Јas]

Or they both do and will soon be regressing to the mean.

Actually, the odds are 1 in 400 that this will happen.

Based on what?

1 in 20 polls is an outlier.  For each poll to be an outlier, that's 1 x 1 to 20 x 20 to 1 to 400.

One appears to be an outlier, one isn't.  For both to be an outlier, the odds are 1 to 400.

First off, let me preface this by saying there's a reasonable chance that I'm completely wrong - I'm no maths whiz - but I don't see how the odds would be 1 in 400.

By my estimate, it's closer to a 1.8% (so almost 1 in 55) chance that both contain 1 outlier.

Reasoning:
If there's a 5% chance that any particular poll is an outlier, then the odds that 1 of a set of 3 polls is an outlier is 3(0.05*0.95*0.95) = 0.135 (that is, 13.5%).
To calculate the chance that both trackers contain an outlier in their sample, I presume one should multiply 0.135 by itself giving a result of approximately 0.018, or a 1.8% chance that both contain an outlier in their 3 day samples.

Anyway, as I say there's a reasonable chance that my reasoning is farcically wrong, but if someone could clarify all this, I'd appreciate it.

[Alcon]

Or they both do and will soon be regressing to the mean.

Actually, the odds are 1 in 400 that this will happen.

Based on what?

1 in 20 polls is an outlier.  For each poll to be an outlier, that's 1 x 1 to 20 x 20 to 1 to 400.

One appears to be an outlier, one isn't.  For both to be an outlier, the odds are 1 to 400.

First off, let me preface this by saying there's a reasonable chance that I'm completely wrong - I'm no maths whiz - but I don't see how the odds would be 1 in 400.

By my estimate, it's closer to a 1.8% (so almost 1 in 55) chance that both contain 1 outlier.

Reasoning:
If there's a 5% chance that any particular poll is an outlier, then the odds that 1 of a set of 3 polls is an outlier is 3(0.05*0.95*0.95) = 0.135 (that is, 13.5%).
To calculate the chance that both trackers contain an outlier in their sample, I presume one should multiply 0.135 by itself giving a result of approximately 0.018, or a 1.8% chance that both contain an outlier in their 3 day samples.

Anyway, as I say there's a reasonable chance that my reasoning is farcically wrong, but if someone could clarify all this, I'd appreciate it.

I believe Jas has the right idea, not to mention that the 1-in-20 rule assumes a perfect sample, or at least consistently-selected samples between two days (in the case between intra-poll comparisons.)

Also, the 1-in-20 rule relates to MoE; it is not a magical "outlier theorem," or anything.  It's the chance, at the 95% confidence rate, that a perfectly-representative poll falls outside the MoE.

[Lunar]

Or they both do and will soon be regressing to the mean.

Actually, the odds are 1 in 400 that this will happen.

Based on what?

1 in 20 polls is an outlier.  For each poll to be an outlier, that's 1 x 1 to 20 x 20 to 1 to 400.

One appears to be an outlier, one isn't.  For both to be an outlier, the odds are 1 to 400.

That's just not true.  If two polls vary significantly from one another, THE MOST LIKELY RESULT IS THE AVERAGE OF THE TWO (putting them both off, which will soon regress to the mean as more polls are taken, Rasmussen up and Gallup down).

[J. J.]

Or they both do and will soon be regressing to the mean.

Actually, the odds are 1 in 400 that this will happen.

Based on what?

1 in 20 polls is an outlier.  For each poll to be an outlier, that's 1 x 1 to 20 x 20 to 1 to 400.

One appears to be an outlier, one isn't.  For both to be an outlier, the odds are 1 to 400.

First off, let me preface this by saying there's a reasonable chance that I'm completely wrong - I'm no maths whiz - but I don't see how the odds would be 1 in 400.

By my estimate, it's closer to a 1.8% (so almost 1 in 55) chance that both contain 1 outlier.

Reasoning:
If there's a 5% chance that any particular poll is an outlier, then the odds that 1 of a set of 3 polls is an outlier is 3(0.05*0.95*0.95) = 0.135 (that is, 13.5%).
To calculate the chance that both trackers contain an outlier in their sample, I presume one should multiply 0.135 by itself giving a result of approximately 0.018, or a 1.8% chance that both contain an outlier in their 3 day samples.

Anyway, as I say there's a reasonable chance that my reasoning is farcically wrong, but if someone could clarify all this, I'd appreciate it.

I'm looking at the odds that there is a "bad" sample.  That is about 1 in 20.  That will screw up three days of polling, but that isn't what I'm looking at.  I'm just interested in the bad sample.  The odds on both happening on the same day are 1 to 400.

You can argue that it would be 1 in 6 2/3 that this be in a three day sample. That two sets of three day samples would both contain a bad sample is above 44 to 1.

Therefore probably both do not contain a bad sample, though one may.  The outlier could be very well be Rasmussen; we'll probably know which one is the outlier by Saturday, if not sooner.

[minionofmidas]

Or they both do and will soon be regressing to the mean.

Actually, the odds are 1 in 400 that this will happen.

Based on what?

1 in 20 polls is an outlier.  For each poll to be an outlier, that's 1 x 1 to 20 x 20 to 1 to 400.

One appears to be an outlier, one isn't.  For both to be an outlier, the odds are 1 to 400.

(Ignoring Alcon's pertinent remarks about perfect samples - which as a rule do not exist in opinion polling - for the moment.) Except that that's a random cutoff point. Not every point outside the MoE is as unlikely as every other point outside the MoE, and not every point within the MoE is as likely as every other point within the MoE.

We're not talking a field divided into a 95% field that is white and a 5% field that is black. We're talking a field that is, almost in its entirety, a very whitish shade of grey, moving into downright white along much of the one half, and into darker shades towards the other end, theoretically reaching black at the very extreme edge.

As to the math...

Yes, two out of perfect samples to be of the 5% share of our whitish-grey field that we whimsically declare to be "black" is a 1-in-400 share. But, as Jas points out, we're actually talking at least one of three and at least one of another three.

If there's a 5% chance that any particular poll is an outlier, then the odds that 1 of a set of 3 polls is an outlier is 3(0.05*0.95*0.95) = 0.135 (that is, 13.5%).

Close, but not quite. That is actually the probability that exactly one of the three is a "bad" sample. The probability that at least one is "bad" is actually 1-(0.95*0.95*0.95) = 0.142625 (the part in the bracket is the probability that all three are "good".)

You must be logged in to read this quote.

Correct.
The actual figure then is approximately 2.0342%, or more than one in fifty.

But of course, all of this really presupposes that pollsters are getting an entirely random sample from the population, are not lied to (or even erroneously given wrong information), and are not performing any mathematical tricks on their raw data to make the random sample's demographics fit their notions of statistical reality.

[emailking]

First of all, an "outlier" is a measurement that is "distant" from the rest of the data. This is a subjective definition, but normally it does not apply necessarily to any result that falls outside of the 95% confidence region on the mean.

Aside from that, Jas and J.J. appear to be using correct mathematics to address different issues. 2.03% chance that both tracking polls have an incorrect poll to within MOE. 0.25% chance that on a given day both tracking polls polled incorrectly to within MOE.

Edit: Fixed prob per Laurent's post.

[Lunar]

Or they both do and will soon be regressing to the mean.

Actually, the odds are 1 in 400 that this will happen.

Based on what?

1 in 20 polls is an outlier.  For each poll to be an outlier, that's 1 x 1 to 20 x 20 to 1 to 400.

One appears to be an outlier, one isn't.  For both to be an outlier, the odds are 1 to 400.

First off, let me preface this by saying there's a reasonable chance that I'm completely wrong - I'm no maths whiz - but I don't see how the odds would be 1 in 400.

By my estimate, it's closer to a 1.8% (so almost 1 in 55) chance that both contain 1 outlier.

Reasoning:
If there's a 5% chance that any particular poll is an outlier, then the odds that 1 of a set of 3 polls is an outlier is 3(0.05*0.95*0.95) = 0.135 (that is, 13.5%).
To calculate the chance that both trackers contain an outlier in their sample, I presume one should multiply 0.135 by itself giving a result of approximately 0.018, or a 1.8% chance that both contain an outlier in their 3 day samples.

Anyway, as I say there's a reasonable chance that my reasoning is farcically wrong, but if someone could clarify all this, I'd appreciate it.

I'm looking at the odds that there is a "bad" sample.  That is about 1 in 20.  That will screw up three days of polling, but that isn't what I'm looking at.  I'm just interested in the bad sample.  The odds on both happening on the same day are 1 to 400.

You can argue that it would be 1 in 6 2/3 that this be in a three day sample. That two sets of three day samples would both contain a bad sample is above 44 to 1.

Therefore probably both do not contain a bad sample, though one may.  The outlier could be very well be Rasmussen; we'll probably know which one is the outlier by Saturday, if not sooner.

An outlier is not a binary distinction.

Two polls showing different results, let's say 8%.  Both polls are reliable and have the same SS.  It is NOT more likely for one poll to be 8% off than for both to be 4% off.  End of story.

Most likely result, and the prediction that will give you the smallest margin of error is to average the results, weighted by sample size.

[emailking]

Two polls showing different results, let's say 8%.  Both polls are reliable and have the same SS.  It is NOT more likely for one poll to be 8% off than for both to be 4% off.  End of story.

The odds of being inside/outside the confidence interval is 95% / 5% by construction of these polls. It doesn't matter how much one of the polls is off by or how they compare to each other.

[minionofmidas]

Two polls showing different results, let's say 8%.  Both polls are reliable and have the same SS.  It is NOT more likely for one poll to be 8% off than for both to be 4% off.  End of story.

The odds of being inside/outside the confidence interval is 95% / 5% by construction of these polls. It doesn't matter how much one of the polls is off by or how they compare to each other.

No, it does. It's a Gaussian bell curve (theoretically). 5% is a random cutoff.

To provide the maths to back up Lunar's thesis would require me remembering maths stuff I learned twelve years ago and haven't used for nine, so I won't try.



   
 
 

[emailking]

Two polls showing different results, let's say 8%.  Both polls are reliable and have the same SS.  It is NOT more likely for one poll to be 8% off than for both to be 4% off.  End of story.

The odds of being inside/outside the confidence interval is 95% / 5% by construction of these polls. It doesn't matter how much one of the polls is off by or how they compare to each other.

No, it does. It's a Gaussian bell curve (theoretically). 5% is a random cutoff.

The MOE is constructed on the basis of the "random" cutoff. If one chose a different cutoff, there would be a different MOE. We're saying that a poll is correct if the actual result is within the MOE. That's a 95% chance by construction. It doesn't matter whre it fits in or how far off it may be. This is statistics 101.

[minionofmidas]

Two polls showing different results, let's say 8%.  Both polls are reliable and have the same SS.  It is NOT more likely for one poll to be 8% off than for both to be 4% off.  End of story.

The odds of being inside/outside the confidence interval is 95% / 5% by construction of these polls. It doesn't matter how much one of the polls is off by or how they compare to each other.

No, it does. It's a Gaussian bell curve (theoretically). 5% is a random cutoff.

The MOE is constructed on the basis of the "random" cutoff. If one chose a different cutoff, there would be a different MOE. We're saying that a poll is correct if the actual result is within the MOE. That's a 95% chance by construction.

Yes.

You must be logged in to read this quote.

For the question of whether it's within MoE it doesn't matter. For the exact probability of an event, it matters as a matter of course. This isn't about the construction "Margin of Error" - this is about the stuff "Margin of Error" is constructed out of.

[Kaine for Senate '18]

Thursday, October 9
Obama: 52 (nc)
McCain: 41 (nc)

[Wiz in Wis]

Thursday, October 9
Obama: 52 (nc)
McCain: 41 (nc)

Delicious

[Zarn]

And Gallup falls from grace.

[emailking]

For the question of whether it's within MoE it doesn't matter.

Well maybe I missed something but as far as I can tell ,that's all JJ was talking about. yes we obviously have to assume the pollsters conducted a perfectly random sampling and were able to measure the results 100% accurately. Neither is true of course, but you can throw out the polls altogether if these effects cannot be assumed to be small.

[J. J.]

For the question of whether it's within MoE it doesn't matter.

Well maybe I missed something but as far as I can tell ,that's all JJ was talking about. yes we obviously have to assume the pollsters conducted a perfectly random sampling and were able to measure the results 100% accurately. Neither is true of course, but you can throw out the polls altogether if these effects cannot be assumed to be small.

You are correct.  I'm suggesting that either Rasmussen or Gallup has a single day sample that is outside of the MOE.  It happens, and it doesn't mean either Rasmussen or Gallup have fallen from grace.  This effect is to be expected. 

We should know which one is wrong by no later than Saturday.  Just chill until then, everyone. 

[Lunar]

Again, think about it.

You have two results that disagree with one another.  The one prediction that minimizes the amount of margin of error is both samples averaged, weighed based on sample size.  If four polls agree and two are way off, you don't just toss out the two, you average all six together to yield the most accurate result.

Again, let's take my two poll, 8% off example.  Both polls will have 3% MoE.  The guess that has both polls at 4% off has two polls polling about 1% out of the 95% confidence interval.  That 1/20 chance outside of the confidence interval does NOT mean that every number outside of that 95% area is equally unlikely, it gets progressively and exponentially worse.  It's more likely to have two polls be, what, half a standard deviation out from the confidence interval away than one poll two standard deviations out.

Not saying you're wrong, but the idea that two polls showing different results would only have 1/400 chance of both being off and having the actual result be in the middle is KARAZEE.

[emailking]

Again, think about it.

You have two results that disagree with one another.  The one prediction that minimizes the amount of margin of error is both samples averaged, weighed based on sample size.  If four polls agree and two are way off, you don't just toss out the two, you average all six together to yield the most accurate result.

Again, let's take my two poll, 8% off example.  Both polls will have 3% MoE.  The guess that has both polls at 4% off has two polls polling about 1% out of the 95% confidence interval.  That 1/20 chance outside of the confidence interval does NOT mean that every number outside of that 95% area is equally unlikely, it gets progressively and exponentially worse.  It's more likely to have two polls be, what, half a standard deviation out from the confidence interval away than one poll two standard deviations out.

Not saying you're wrong, but the idea that two polls showing different results would only have 1/400 chance of both being off and having the actual result be in the middle is KARAZEE.

Yes, that would be crazy. But you changed the problem. You are now presupposing what the true answer is that is being measured by the sampling in your analysis.  If we already know both polls are wrong then there is a 100% chance they are both wrong. But the scenario you layed out has a 1/400 chance of happening.

Since a priori we know neither the answer nor where the distributions lie, then the best that can be done is to treat them as independent events.