A few new state polls...

[struct310]

The AZ poll was done by ASU which always gives Kerry a good showing in polls.  

[Smash255]

pssst....  hey vorlon, chambliss won in georgia.

(note to self - learn to read, get tested for dyxlexia)

You know, this is the first time I have ever taken a hard statistical look at Zogby.  I just typically ignore him as just too eratic.  He is actually really bad in a lot of ways.

I have always known he is erratic, but when you put hard numbers on it it is kinda scary actually.

In his last 33 polls he has missed by more than 6% 12 times. (!)

I may replace his current logo in the polls section:



with a new one:



Yikes !

Zogby was off in 02 with mainly senate polls, but in 2000 he was very accurate in the Presidentialrace.  Had Fore winning the popular, but losing electoral

[The Vorlon]

Zogby was off in 02 with mainly senate polls, but in 2000 he was very accurate in the Presidentialrace.  Had Fore winning the popular, but losing electoral

I was unaware that Zogby made a formal EV projection.  Given he called Florida for Gore, Michigan, and Pensylvania for Gore, I think him calling the Electoral College for Bush is profoundly unlikely.

Please provide a linkto Zogby's EC projection showing a Bush win.

For the Record, Zogby had Gore at +2 for his final popular vote projection, The final popular vote at 48/48 gave Zogby an error of just 2% which was excellent.

But at least 6 other polls were actually closer. The pollsters had a great year in 2000.

Harris Interactive (Internet) projected Gore + 0.6%
Harris (Phone) projected a tie
Fox News Projected a tie
Snell/Perry/Lake had Gore +1
CBS (!) projected Gore +1
Yerxa/DDC had Gore +1

Finally, others had Bush at +2, essentially tying Zoggy for accuracy, at 2% error to the other side of things - missing to the Bush side not the Gore side:

TIPP Had Bush + 1.9%
Gallup had Bush +2

This claim that Zogby (Made by many, not specifically you) was the most accurate in 2000 is just wrong - He was tied for 6th among the polls I am aware of.

Zogby's state polls in 2000 were also all over the place.

His Final state poll in California had it tied believe it or not! (an error of 11% !)

He missed New York by 6%, Illinois & Pennsylvania by 5%,

In 2000 on the Senate side he missed New York by 10%, Pennsylvania by 9%, and Washington by 5%....

To be fair, in 6 of the 11 presidential state races he polled in 2000 he was within 3%, but on 5 out of 11 he was not...

For the record, in 2002 the National Council on Public Opinion Polls analysed all the pollsters and found that Zogby had both the highest average error from the actual result and the highest % or races not correctly predicted of any firm

http://www.mason-dixon.com/news_text.cfm?news_id=158

Zogby has nailed more than his share of races, he has also utterly screwed up way more than his share of races too.

I am in the process of organizing all my polling stats and Zogby is profoundly erratic.

In his last 33 published polls he has missed by more than 6% a dozen times (!)

[afleitch]

Perhaps this should be renamed the anti-Zogby thread! I think you're all getting worked up over nothing. Yes the state polls are meaningless, but look at it this way. Polls showed Iowa as being erratic of late, a bit like Wisconsin in April, before it 'settled' down in favour of Kerry. Zogby says Bush is ahead- thats the way the tide seems to be going at this rate. And Missouri- a state that has all but been forgotten by polsters since March seems to be in the Kerry column- maybe it actually is! No ones touched the state for months! And no offence intended, but just because Vorlon says something, doesn't mean you all have to take it as the word of god! Being bogged down in 'analysing the analysis' of polls can make you miss what is actually happening. Figures aside, perhaps Iowa is swinging to Bush and perhaps Missouri and Nevada are swinging towards Kerry for now. We don't know and armchair analysts on this board don't know- the people with access to the best methods, even if you don't like them, are the pollsters. Don't chastise them for giving you figures you don't like!

[millwx]

Also, in case you've missed it... a new Quinnipiac poll from PA just released today!

Kerry 45%
Bush 42%

...or, in a three way race....

Kerry 44%
Bush 41%
Nader 6%

The poll was conducted May 24th-25th of 701 RV

[The Vorlon]

And no offence intended, but just because Vorlon says something, doesn't mean you all have to take it as the word of god! !

Damn Good point!  Hear hear!

(Wait what am I saying I am a god.. 577 posts... ok...different type of God... )

By November we will have 6 or 7 or 8 different daily tracking polls going of the national race, probably 3 or 4 different sets of state level presidential tracking polls, and god knows how many individual state polls.

There is a shortage of many things in this world (wisdom, kindness, courage to name a few) but polls is not one of them

If you don't like a particular poll, average the other 23 available choices.

But afleitch is right, this far out, lets stop beating up one poll or one set of results.

I hereby apologize, I got on a bit of a rant on Zogby - consider it a family fued - the "hard weighters" versus the hard core "probability samplers" - it is the polling community equivalent of the Hatfields and the McCoys.

My apologies for disrupting the decorum of the Forum!

(hangs head in shame - slinks away.. )

BTW, the WSJ just published a bunch of new TELEPHONE Zogby polls of various states.

The polls are often wrong.  

That is why we still have 110+ million people actual go to the ballot boxes and go through the time and expense of counting up all the little bits of paper.

Reagan blew out Carter in 20001980, everybody got it wrong.

Sometimes things happen that nobody sees coming... that's what makes it fun

Good post afleitch !!!

[classical liberal]

Reagan beat Carter in 1980, not 2000.

[Giant Saguaro]

Also, in case you've missed it... a new Quinnipiac poll from PA just released today!

Kerry 45%
Bush 42%

...or, in a three way race....

Kerry 44%
Bush 41%
Nader 6%

The poll was conducted May 24th-25th of 701 RV

Another Q poll? The one I saw was 45-45 and 43-42, Bush, with Nader in the race. That could have been LVs, though. Either way, there's little difference.

Oops, I think this si a state poll, not a national poll. If so, sorry, posted this in error.

[afleitch]

Thank you Vorlon for such a nice reply! I do usually think your spot on with your predictions, but I'm glad you agree that too many people salivate at your feet! I think everyone should experiment with their own polling preferences. I wouldn't trust Rasmussen as close I could throw it when it comes to predictions, but they are good for trends, which at this stage mean a helluva lot.

[The Vorlon]

Thank you Vorlon for such a nice reply!

Hey I'm a nice guy

I am putting together what I "hope" will be an objective and generally accepted way of measuring pollsters for use in the Polls part of the forum, and would like your feed back.

The first thing I did was project out what a "perfect" pollster would look like.

I projected based on pure probability what % of the time a sample size of 1000 would report a candidate lead within the following brackets:

Within 2% of actual
Between 2% and 3% error from actual
Between 3% and 4% error from actual
Between 4% and 5% error from actual
Greater than 6% error from actual

I rather arbitrarily used the old University grading scale and assigned polls a mark

Within 2% = "A" = 4.00
Between 2% and 3% = "A-" = 3.7
Between 3% and 4% = "B"= 3.0
Between 4% and 5% = "C"= 2.0
Between 5% and 6% = "D"= 1.0
Greater than 6% = "F"= 0.0

A methodologically "perfect" poll that had only random chance as it's only source of error would over many, many polls get a score of 3.22 under this system, so I then introduced a "Fudge factor" of multiplying the final score by 1.24 so a "perfect" pollster would get a "perfect" score of 4.00.

I then take all the polls from a firm I can find, average their total poll grades , multiply by 1.24 and voila! - you get a pollsters "GPA".

There are a few kinks I am notquite sure how to deal with fairly.  

For example Mason-Dixon typically uses a smaller sample - usually 600 or 625, so if they were methodolugically perfect their score would be a  bit lower than 4.00 (3.48 actually) simply because their margin of error is a bit bigger.   So M/Ds grade would be a bit lower.  But then if M/D did do a poll of say 1000 sample size, their "grade" would understate how accurate their 1000 sample size poll is likely to be.

Another is the cutoff dates to use.  Obviously if you poll a state in July, I am not going to hold it against a firm if it turns out wrong in November, so I have basically pulled 10 days before the actual vote date out of my hat as the cutoff.

Another issue is Primaries.  Primaries are such a total nightmare to poll I have decided not to use them at all simple because, over time, everybody would fail.

Not perfect, but on balance, I think it is a good approach.

Essentailly what will come out would look something like this:

Polling Firm "X"
     -Total Polls graded = 36
     -GPA: 1.69 = C-
     -Polls within 3% = 11/36 = 31%
     -Polls between 3-6% = 13/36 = 36%
     -Polls with error > 6% = 12/36 = 33%

I have only done 7 firms so far (I have set a minimum of 20 polls to get a grade) and generally speaking firms have performed about how I expected, but a few surprises too.

Research2000 was in my mind about a B- firm, but statistically are a bit better than that.

Mason-Dixon is, statistically speaking, rather spectacular actually, performing slightly better than statistically expected.

Public Opinion Strategies is just blowing me away with how good they have been the last 3 election cycles.

Gallup while still very good, is not quite as good as I expected.

It is always nice to do a validity check of one's own impressions versus hard actual data, and correct where needed.



Take care!

[Gustaf]

I knew Gallup was a scam!

I am biased against Gallup, since their Swedish branch sucks so much. They missed our nation-wide referendum last year by 15 percentage points!.

[millwx]

would report a candidate lead

As you know from my off-forum e-mails to you, I have a real problem with the actual percentage the lead candidate gets, because varying undecided percentages can water down the raw percentage.  So.....

1) I like your idea above of using the "candidate lead" rather than actual percentages.  Barring a huge candidate lead (which, I suppose, is possible sometimes), this number will be far less watered down by undecideds than the raw numbers.  And...
2) I think you're wise to stay close to the election (10 days), when undecideds are low.  The undecideds are an additional reason (besides the obvious one you stated... you can't expect a poll 6 months in advance to be right) to keep the time frame short.

Your idea may be imperfect, but it looks decent to me... good enough, at least, to provide some objective guidance on poll quality.

[The Vorlon]

I knew Gallup was a scam!

I am biased against Gallup, since their Swedish branch sucks so much. They missed our nation-wide referendum last year by 15 percentage points!.

15% -

What was the question, and was it a "stand alone" referendum or was it part of a general election?

15% error that is a really amazing level of error.

[Gustaf]

I knew Gallup was a scam!

I am biased against Gallup, since their Swedish branch sucks so much. They missed our nation-wide referendum last year by 15 percentage points!.

15% -

What was the question, and was it a "stand alone" referendum or was it part of a general election?

15% error that is a really amazing level of error.

The issue was adoption of the euro. And it was "alone", the only issue being voted on. Turnout was the same as in normal electinos, about 80%. Gallup polled it throughout the entier campaign ratehr frequently. The last poll they published, the day before the referendum showed the yes-side ahead, 43-42 with the rest undecided. The final result was a no-win, 56-42 with 2% abstainees.

They changed the exact wording of their question in their last poll, probably to help the yes-side (there was a lot of that since the entire political/economical establishement was on the yes-side).

[The Vorlon]

I knew Gallup was a scam!

I am biased against Gallup, since their Swedish branch sucks so much. They missed our nation-wide referendum last year by 15 percentage points!.

15% -

What was the question, and was it a "stand alone" referendum or was it part of a general election?

15% error that is a really amazing level of error.

The issue was adoption of the euro. And it was "alone", the only issue being voted on. Turnout was the same as in normal electinos, about 80%. Gallup polled it throughout the entier campaign ratehr frequently. The last poll they published, the day before the referendum showed the yes-side ahead, 43-42 with the rest undecided. The final result was a no-win, 56-42 with 2% abstainees.

They changed the exact wording of their question in their last poll, probably to help the yes-side (there was a lot of that since the entire political/economical establishement was on the yes-side).

Interesting, I wonder if sweden's Gallup is the same organization as US Gallup, or not.

In the US, since 1936, the average Gallup "bias" has been 0.33% to the Democratic side, so over on this side of the pond they are pretty darn straight.

(As opposed to 3.68% bias to the democratic side for an "unnamed" poll associated with a particular TV Network)

(Stop it.. promised aflietch I'd play nice...)

15%, especially when you have a pretty decent idea what turnout will be is just about impossible to do unless you have something really, really screwed up and/or tampered with in your methodology.

What was the sample size - 8, 12.. ?

[Gustaf]

I knew Gallup was a scam!

I am biased against Gallup, since their Swedish branch sucks so much. They missed our nation-wide referendum last year by 15 percentage points!.

15% -

What was the question, and was it a "stand alone" referendum or was it part of a general election?

15% error that is a really amazing level of error.

The issue was adoption of the euro. And it was "alone", the only issue being voted on. Turnout was the same as in normal electinos, about 80%. Gallup polled it throughout the entier campaign ratehr frequently. The last poll they published, the day before the referendum showed the yes-side ahead, 43-42 with the rest undecided. The final result was a no-win, 56-42 with 2% abstainees.

They changed the exact wording of their question in their last poll, probably to help the yes-side (there was a lot of that since the entire political/economical establishement was on the yes-side).

Interesting, I wonder if sweden's Gallup is the same organization as US Gallup, or not.

In the US, since 1936, the average Gallup "bias" has been 0.33% to the Democratic side, so over on this side of the pond they are pretty darn straight.

(As opposed to 3.68% bias to the democratic side for an "unnamed" poll associated with a particular TV Network)

(Stop it.. promised aflietch I'd play nice...)

15%, especially when you have a pretty decent idea what turnout will be is just about impossible to do unless you have something really, really screwed up and/or tampered with in your methodology.

What was the sample size - 8, 12.. ?

I am pretty sure it's the same company. It's an international firm. The sample size was your regular 1000... But I learned not to put too much faith in sample sizes...the firm with the biggest sample sizes in Sweden, SKOP, really sucks. Of course, a sample size of 250, as one major pollster used in their final poll, might mean you miss the result by 14 percentage points...Gallup wasn't all that alone. 3 out of 5 pollsters had a tie in their final polls...