JFern's "Statistics"

[ATFFL]

If the actual population is 51% Bush, 49% Kerry, the odds are extremely small that one of 100 polls would give something like 94% Kerry, 6% Bush. In fact, the MOE is pretty small on a 94% Kerry, 6% Bush poll (it would be 0.75%)

Are you saying that the MOE is determined by the results?  Or did you mean to say the chance of a poll showing 94-6 when the actual population is 49-51 is .75%?

Also, what happened on Electoral Vote.com is that when ARG came out with their polls the method the site used predicted a Kerry win.  All they did was assume that whatever movement occured from the last two polls would continue indefinately.

[ATFFL]

If the actual population is 51% Bush, 49% Kerry, the odds are extremely small that one of 100 polls would give something like 94% Kerry, 6% Bush. In fact, the MOE is pretty small on a 94% Kerry, 6% Bush poll (it would be 0.75%)

Are you saying that the MOE is determined by the results?  Or did you mean to say the chance of a poll showing 94-6 when the actual population is 49-51 is .75%?

I'd appreciate this getting clarified.

[ATFFL]

That calculator uses 1.96 standard deviations for its 95% confidence interval. Look at the source if you don't believe me. I see you haven't gone to that website, you g fraud. 

You didn't quote the part where I talked about this link. I bet you hope this site will go away. Sorry, it doesn't work that way.
http://americanresearchgroup.com/moe2.shtml

J. Idiot., you lose. Argument over.

Quick question:  How does that calculator help if the sample is not a representation of the public at large?

[ATFFL]

That calculator uses 1.96 standard deviations for its 95% confidence interval. Look at the source if you don't believe me. I see you haven't gone to that website, you g fraud. 

You didn't quote the part where I talked about this link. I bet you hope this site will go away. Sorry, it doesn't work that way.
http://americanresearchgroup.com/moe2.shtml

J. Idiot., you lose. Argument over.

Quick question:  How does that calculator help if the sample is not a representation of the public at large?

It only works if you assume a random sample.

I thought the goal was to get a randomly selected representative sample.  If I poll a random selection of 1000 Massachusetts Dems and get a 96-4% result it may be statistically significant and completely useless.

[ATFFL]

That calculator uses 1.96 standard deviations for its 95% confidence interval. Look at the source if you don't believe me. I see you haven't gone to that website, you g fraud. 

You didn't quote the part where I talked about this link. I bet you hope this site will go away. Sorry, it doesn't work that way.
http://americanresearchgroup.com/moe2.shtml

J. Idiot., you lose. Argument over.

Quick question:  How does that calculator help if the sample is not a representation of the public at large?

It only works if you assume a random sample.

I thought the goal was to get a randomly selected representative sample.  If I poll a random selection of 1000 Massachusetts Dems and get a 96-4% result it may be statistically significant and completely useless.

Well, a random sample is usually pretty representative.

But not always.  That is why we do validity checks.  Would you accept my above poll as representative of the nation?  Even if they were somehow chosen randomly from the entire US population?

Even if you get the random representative sample there is still a chance that reality is outside of your Margin of Error.  If you do not have a proper sample to begin with, you have error on top of error.  Sometimes two wrongs do make a right, but not usually.

[ATFFL]

You're asking for a weighted random sample. That would be less random then a random sample. If you knew you had the true weight (let's say 35% Dem, 35% GOP, 30% Indepdendent, or something, I made those numbers up), then you can calculate the MOE on each part equally, and find the combined MOE weigted MOE, by finding the MOE of each subsample, and adding them using the Pythagorean formula for adding standard deviations.  Then, that would give you a smaller MOE, but it's not what I was talking about because didn't want to get into complicated matters like weighted non-random polls.

No, weighting is not the same thing as I am talking about.  I am talking about validity checks.

Simple question:  Is a poll of 1000 registered Democrats livining in Boston representative of the general electorate?

Random samples can and  blow up.  They can pull in far too many members of any group.  We know from the census that the US population has 12.9% African Americans.  Let's say that If a poll gets 20% African American respondednts that is too much for the nation.  We can either weight it down to a more reasonable amount, increase our sample size by interviewing people of other ethnic groups, toss the poll out and start over or, if we are irresponsible, release it and pretend it is a propre representative sample.

If we get a poll that is 14% African American we can weight it down, or leave it be since it is close enough.

A poll can be picture perfect, spot on for every measurable demographic and perfect in every technical aspect with questions that are perfectly neutral and readers who show no prejudice one way or another and still be completely wrong.

[ATFFL]

jfern, your knowledge of theoretical statistics cannot be doubted, but your knowledge of applied statistics is lacking in this case.

You are assuming that the polls are measuring random variables.  Who people vote for is not random.  They do not take a perfect coin and flip it to decide who to vote for.  People decide based on a vast number of variables that, even in a dead even 50-50 race, is not something that can be perfectly measured.

If we were arguing about coin flips, you would be spot on perfect correct about nearly everything you have said.  Sadly for you, we are not talking about coin flips.

This is why pollsters need to check validity.  For a coin tossing experiment all we need to do is make sure we a have a perfect coin, one weighted so neither heads nor tails is biased and to make sure we do not bias the toss by not properly spinning the coin when we toss it.

For polls of people we have to run a lot more checks.  We need to make sure our sample is sufficiently random.  Then we need to make sure our random sample is representative of the population as a whole.  Then we need to make sure our questions and readers were unbiased.  Then we need to run our results through a "smell test" to see if they are obvious outliers compared to other results from a similar time frame.  If there are no similar results we have to decide if the numbers are so obviously bad releasing them will hurt our reputation as a pollster. 

Let’s say we get that poll showing Kerry up 96-4.  Is that a statistically significant lead?  Of course it is.  Does that mean that Kerry is leading by that much?  Of course not.  We still need to check if the poll is valid by seeing if we have a random sample, a representative sample and is not an obvious outlier. 

If the electorate is tied 50-50 and we get that 96-4 result we know that our result is garbage and need to toss it.  Even if it is a representative random sample showing a massive, statically significant lead, it is going to fail the last test.  When every pollster shows the race at or near 50-50 but us, odds are we are the ones blowing it. 

Now if we do a poll of African Americans who are registered democrats living in Washington D.C. that 96-4 result begins to look a lot better. 

If our polling was done in a theoretically perfect world where voters decided how to vote based on perfectly random means you would be spot on correct.  We do not live in that world.