On summer polls....

[Gustaf]

Since Vorlon is no longer here I thought I would remind some newbies of his old words of wisdom: "All summer polls that aren't Mason-Dixon should be immediately burned". (or something to that extent)

They tend to a) favour Democrats and b) be generally wrong.

[© tweed]

...but they give us something to talk about.  Good enough for me.

[True Democrat]

Why do they favor Democrats again?  I never really understood this.  Does it ahve to do with the teachers being out or something?

[CARLHAYDEN]

Why do they favor Democrats again?  I never really understood this.  Does it ahve to do with the teachers being out or something?

Actually, evidence tends to indicate that Republican leaning voters are less likely to be home during the summer to answer telephone polls.

[Alcon]

As CARLHAYDEN said, I think the reason is a compound of Republican families being more likely to take vacations and Republicans being more likely to be employed in an occupation that requires work during the summer months.  Students are also home.

I'm not sure Mason-Dixon does anything special to adjust for this (it would be hard).  It may just be that they are so accurate that, even if there is a slight Dem lean during summer months, it doesn't much matter.

[Gustaf]

Though no one seems to want to say it out loud, people with more money tends to go away on vacation when they're not working, while people with no money can't afford to do so and therefore sit around at home. Needless to say, the rich tend to vote more Republican, hence, Republicans get under-sampled in the summer.

Mason-Dixon is just that good, I think.

[True Democrat]

Well, this trend should be lessening and lessening.  People with money are trending Democrat.

[Tender Branson]

When does Mason Dixon ever bring out a poll ? Once in 3 months ? I don´t know the most accurate pollsters but i have to say like the previous poster: We need a lot so we can discuss them.

[© tweed]

When does Mason Dixon ever bring out a poll ? Once in 3 months ? I don´t know the most accurate pollsters but i have to say like the previous poster: We need a lot so we can discuss them.

They don't poll much early in an election season but usually release a boatload at the end.  Look at what they did in 2004:

link

They basically polled every battleground state a few days before the election.  They did very well too.

[ATFFL]

Want a mason-Dixon poll?  pay them.

[Eraserhead]

Since Vorlon is no longer here I thought I would remind some newbies of his old words of wisdom: "All summer polls that aren't Mason-Dixon should be immediately burned". (or something to that extent)

They tend to a) favour Democrats and b) be generally wrong.

Give me a break. Do you work for Mason-Dixon?

[Jake]

He's smart is all.

[Alcon]

Since Vorlon is no longer here I thought I would remind some newbies of his old words of wisdom: "All summer polls that aren't Mason-Dixon should be immediately burned". (or something to that extent)

They tend to a) favour Democrats and b) be generally wrong.

Give me a break. Do you work for Mason-Dixon?

If he works for Mason-Dixon, we all work for Mason-Dixon.

[Sam Spade]

Since Vorlon is no longer here I thought I would remind some newbies of his old words of wisdom: "All summer polls that aren't Mason-Dixon should be immediately burned". (or something to that extent)

They tend to a) favour Democrats and b) be generally wrong.

Give me a break. Do you work for Mason-Dixon?

Clearly, you need to start engaging with reality a little more.

Vorlon is the most knowledgable person concerning political polling here of any of us, more than me, Gustaf and Alcon, and a few other nonpartisans put together.

This knowledge derives from the fact that he works within the field and has actually conducted polling for a congressional campaign or two.

Mason-Dixon is ok during the summer, because Mason-Dixon does neither hard weight polling (a la Zogby) or unweighted polling (a la SUSA), but rather focuses its surveyors to gather up a sample representative of a state on a combination of geographical factors and party representation.  There are also other reasons, but I won't list them here.

Someone should bump Vorlon's thread concerning polling if they can find it.

I will also note to you that Vorlon said that Santorum was toast about 12 months ago.  Of course, he also said Kerry was going to win the election for a quite a long while, but correctly predicted Bush was going to win Florida by 5.

He has said that PA and FL are the two states that he really understands (in a comprehensive way) concerning polling, so I bring up these examples for a reason.

[jokerman]

Polls always skew democratic because liberals are obviously more likely to pick up the phone when it rings.

[Sam Spade]

I'm just posting old Vorlon posts which are fairly relevant, even though they deal with Presidential elections, rather than Mid-term elections.

This is from March 27, 2004.

More of State polls this far out....

(reposted from about a month ago)

The question of what type of turnout helps which party is a little more complex than you suggest...

The really really hardcore base vote is strongly democratic.  If turnout is below say 42-44% or so the GOP is in HUGE trouble, as the Union members, activists, government employess, & government dependant individuals are the most reliable of all voters.

The last 10 years or so has seen the GOP and Democrats in something very close to parity at the House and Senate level.  The change from the years prior to 1994 and today is actually not that the Dems have gotten fewer votes, or that the GOP is more popular - the Dem base has stayed about the same - it is actually that the GOP "Ground Game" has gotten much, much better, and raised GOP turnout.. (the Dems are STILL better at it than the GOP, but the GOP is at least in the game)

When turnout is right around 48-50% or so, this tends to favour the GOP, as the next most reliable set of voters tends to be part of the base GOP constituency.

Finally, when turnout get to be 53+% or so things swing back to the Dems because the occasional or intermittent voters, when they choose to vote, tend towards the Democrats.

The above voter patterns/trends make it very very hard to do accurate polling.

Prior to the Voter Fraud Promotion & Enablement Act "Motor Voter" act, polling was much easier - If somebody took the time and effort to actually go down to their local courthouse and register to vote, they were a likely voter.  You asked somebody if they had registered, and if they had - you counted them in the survey results.

Because of "Moter Voter" the number of people registered to vote has gone up dramatically, but most of the newly registered simply don't vote.  Indeed overall turnout still seems to be at best stable and if anything modestly trending downward.

To try to limit their sample to the 50ish% who actually vote, pollsters ask a whole bunch of screening questions, such as how carefully the voter is paying attention, do they know where their poling place is, did they vote in the last election, how enthusiastic they are about their candidate, etc...  Depending on the firm doing the poll there is a screen of anywhere from 7 to 13 questions. 

Right now the hard core vote, which favours the Dems, is far more tuned in, and hence the sample of "likely" voters is skewed towards the Democrats - Most surveys from now till roughly the conventions will thus have a systemic Democrat bias of somewhere  around +/- 4-5 points.

Somewhere around the Conventions, the GOP base starts "tuning in" to the race, so the number of "likely" voters will increase to the point that the surveys will then skew towards the GOP - indeed most surveys from the conventions till say the end of Septemeber/early October or so will have a pro GOP bias of about 3-4 points or so.

Finally, in late October/Early November the "intermittent" voters start to actually tune in, and the opinion surveys actually start to mean something.

The cycle I have discribed above also explains why the cnn/gallup/usa today poll bounced around like a yo-yo in 2000 (I think there was a 4 day stretch where Bush went from down 13 to up 11) - The gallup is designed to measure things JUST before the election, so any big campaign event which tends to "excite" a particular candidates voters will show a huge and unrealistic spike if the survey is still months or weeks away from the actual vote date, as voters of one party or the other are 'excited" enough to be deemed likely voters..

For example, in 2000 when Bush "won" the first debate he got something like a 10 point bump in Gallup, which was clearly at odds with reality..

[Sam Spade]

April 5, 2004

I thought it might be useful to remind us about polling.

And as an after thought....

Polls go really haywire in July and August when folks are on vacation.  The barely work now, they really do off the deep end when the kids are out of school.

The good firms will compensate, but from the 2nd tier and below ones... expect some truly strange results...

In August if you see Kerry with in the margin of error of Bush in Idaho, or Bush "closing fast" in Rhode Island... just remember this post...

[Sam Spade]

Hey..... You don't have to convince me, of all people, how shaky University polls are..

I think I am "fairly" clearly on the record on that matter.  Quinnipiac is the best however of an admittedly very shaky lot...

Actually, Nader getting 8% is an artifact of a very loose voter screen (this poll is REGISTERED voters not likely voters)

Notice how in the good polls (Gallup & Battleground for example) including Nader changes things by "about" 1 point in Bush's favor?

(Nader getting 1% seems "about" right IMHO) while in some polls including Nader changes things by 3 or 4%.. ?

the reason is very simple..

One very useful question that a few of the better polling firms use is something called the "unaided ballot," where the person is asked who they are going to vote for, but not given any names to pick from, ie the person has to volunteer an answer..

This is a cheap and dirty way to guage real candidate support.  A lot of people when given a list will kinda sorta pick a candidate, but in reality are undecided, not motivated, informed, or aware enough to vote...

Lets be rational here, if you cannot, unaided, actually name the candidate you are going to vote for, how "likely" a voter are you really...?

This is why Nader polls 5 or 6% - those people who are so tuned out the cannot even name Kerry or Bush as their candidate when presented with a list of three names will randomly pick Nader just so they have an answer to the question...

Registered voter polls including waaaaaay too many people as being "likely" - Turnout in 2000 was about 54.7%, the folks who know what they are doing think that 2004 might barely hit 60%

It is by virtue of including a bunch of tuned out folks in the poll that Nader gets 6%

If there is a "big" gap between the result with and without Nader, the poll is just simple casting too wide a net and letting a lot of tuned out people who won't actually vote.

Nader will net out costing Kerry maybe 1% or so, unless Bush has it "in the bag" on election day when a bunch of Deaniacs may protest vote for Nader.

[Sam Spade]

April 26, 2004

Wow... A question I might be able to answer..

A poll has two types of error.

Systematic error:

Systematic error is when there is just something "wrong" with the way the poll is constructed. 

Probably the most common type in here is an improperly drawn sample.

To pick on one of the polls I love to hate, CBS news does somethng structually, fundementally wrong in that they use pure simple "random digit dialing" to get their 1000 or so phone numbers.

What is so bad about this you may ask....?

In a "perfect" sample ever VOTER (not phone number) would have an equal chance of being contacted.

Married couples (ie two votes) tend to have one phone number.

Single people living alone also tend to have one phone number.

Married people break strongly for the GOP, while singles break strongly for the Dems.

Because the ONE single voter has ONE phone line, while the TWO married voters have ONE phoneline, the single voter is twice as likely to be contacted as the married voter..

So CBS news structuarally, inherently, as a matter of poll design, has twice the chance of reaching single people (a strongly pro Democratic voting block) as they do of reaching married people (a strongly pro GOP block)

Guess what - CBS polls always wildly favor the Democrats... this is not a fluke... it is a structural flaw with their poll...

Of course this is a huge over simplification, in reality you have to take into account "second" lines for the teenagers, cell phones, etc... but the illistration is valid...

(Enroll in "Advanced Polling 623" - "how to draw a representative phone sample" - for more information)

A second structural error is a bad or biased question wording.

The classic example of this is the pro life / pro choice polling. (A poll that is, IMHO, just about impossible to do accurately)

For example a question asking if you :

"Favored the continued state funded and sactioned slaughter of unborn children"

will likely produce a somewhat different result that asking:

"Should the oppressive authority of the state be used to force women against their will to bear children"

Both of the above are examples of structural or systemic error.

Random Error

The second type is just "random" error due to sampling, it has not bias that favors one side over the other.

If you flip 100 coins in the air you don't get 50 heads and 50 tails, you usually get 53/47 or something like that, the result is no more likely to favor heads than tails.

Lets use an example of a poll that is 53% for A, and 47% for B with a margin of error of +/- 3%

There is a 68% chance that "A" is between 51.5 and 54.5%

There is a 68% chance that "B" is between 45.5% and 48.5%%

In this example, there is a 95% chance A is between 50 and 56%, with a similar 95% chance "B" is between 44% and 50%

As a very rough rule of thumb, if the "gap" between candidate A and B is less than half of the margin of error the two are pretty much tied.

If A beats B by more than 1/2 the MOE A is likely ahead by a bit

If A beats B by more than the MOE you can be pretty sure they are ahead... 19 times out of 20 anyway..

Typically, Systemic errors, in all but the very best polls, are larger than random errors.

Systemic errors are why there are University polls showing Bush +4 in New Jersey.

This concludes the introductory lession of Polling 101 - are they any questions...?

[Sam Spade]

On Florida, from April 29, 2004

NOTE ON FLORIDA POLLS

Florida is also a very tough state to poll.

1) Huge number of military families stationed overseas have florida as their official residence (no state income tax!)

2) About 18% are unable to (officially anyway) vote due to non-citizen status.

3) Many Seniors who live part of the year in Florida, part elsewhere may, or may not be Florida residents for voting purposes.

4) About 4% of the populaton as felons/ex-felons cannot vote.

5) Language issues with the large cuban/hispanic blocks can make polling difficult from a logistic point of view.

Be VERY careful which polls you look at about Florida.

Mason-Dixon was started in Florida, and is still largely based in Florida, and frankly have an almost god-like reputation for being just about the only ones able to consistently poll the state accurately.

[Sam Spade]

An interesting and observant question....

Rasmusssen is doing a hard weight in his daily tracking poll to a predetermined distribution of DEms/GOP/Ind which will tends to give you both a very stable result, and one which will not drift too far from dead even...

re one day polls...

One day polls have some inherest challenges built into them.  One of  the obvious of these is that you can't talk to people who are not home.

Normally when you do a poll, you select your sample of XXX phone numbers, and then if you do not reach a certain phone number, you try back tomorrow, or the day after till you have reached as much as you can of the original sample, and then only after you have failed to reach a phone number many times, do you substitute in a replicate number. (You usually try a number about 10 to 12 times before you give up and replace the number)

This compensates for the fact that different demographic groups have diferent social, travel and holiday patterns.

The elderly, for example, go out partying far less on Friday night than to 21 year olds, Republicans tend to leave the area more on weekend vacations, etc...

To use a simple example, imagine you did a poll in Green Bay Wisconsin on the day of a Packers game about building the Packers a new stadium.

 A very meaningful chunk of Green Bay Foorball Fans (whom we presume may disproportionately favor a new statium) are at the football game, and hence a Sunday afternoon poll would not reach them - hence your sample would not be a valid snapshot of the town...

If you called back all the numbers where you got no answer on Sunday afternoon by doing "call backs" on Monday, Tuesday, etc till you reached all your original sample, you could get around this problem.  Obviously a one day snapshot poll makes this impossible.

(This is the same reason why the "snapshot" polls some networks try on the night of the Presidential debates are deeply problematic as well...)

I am assuming Rasmussen is building in some systemic counter weights to this "one day" problem by weighting his sample upwards in terms of the number of boomers, Rebpublicans, etc...

How and to what degree he is doing this I do not know.  Perhaps his new "premium" service will shed some light on this..?

[Sam Spade]

May 24, 2004

-----Original Message-----

Subject: A question about polls during the summer.

Hey Vorlon,

This is Wildcard from the Atlas forums and I had a question for ya.

Is it true that the Republican base in polls normally gets weaker during the summer? I've heard this a few times and I was wondering if  it's true or not.

Thanks man and have a good one!


Standard Summer State Poll warning.. again....

If a summer state poll does not say "Mason Dixon" - you should burn it...


Here is a quick reply I am “recycling” – hope it answers your question…

<<cut and paste from another email>>

A real quickie answer…

The “short” answer is, generally speaking, "yes"

Typically the GOP will under poll by 3-5% in the summer. (Democrats under poll by about 3% in late September/early October BTW)

The degree to which this is true depends heavily however on the quality of the firm doing the poll.

To a lesser degree it also depends on the nature of the voter screen used.

As a very broad generalization, the worse the poll, the more the GOP will under poll, the better the firm the less the GOP will under poll. – with a few exceptions I have noted below.

The "correct" answer is a tad more complicated.

A little bit of background.

Many of the lesser quality polling firms use "random digit dialing", (a fancy way of saying that they have a computer randomly pick phone numbers to call) to select who they talk to for a poll. 

This is a flawed way to do a poll anyway unless you use something called "sample stratification" as per Zogby, ABC, SUSA etc (a whole other topic which I will leave for another day), but presents extra problems in the summer because Republicans tend to be on vacation, out of town, away for the weekend, etc a bit more than Democrats. - Obviously not being home, they are contacted less often by a pollster, which skews the sample….

A "good" firm will compensate for this with "call backs" but often the polling firms used by media outlets are selected because they are cheap, rather than because they are good.

Here is an extreme example of this effect.

Assume for a moment that the City of Green Bay (Population 250,000 or so?) had a ballot initiative to build the Packers a new football stadium, you were conducting a poll on this initiative, and that for some reason you decided to make all your calls while the Green Bay Packers were actually playing a home game ( ie 60,000 Packer fans were at Lambeau Field and thus not at home….)

Obviously the sample of people you would be able to actually reach would be very different than the actual population you were trying to sample.  60,000 Packer fans are at the game, and hence you can’t poll them… - as consequence your poll would likely be rather badly wrong.

On way to compensate for this is to do "call backs"

Rather than just keep dialing till you got your designated sample size, you would try everybody on your list once, and then stop. 

The next day you would try again the people you could not reach on day one, and keep retrying on different days and at different times till you reached you entire original sample.

This way, a representative slice of Packer fans that were at the game would be included in your sample when you tried them on Monday, Tuesday, Wednesday….

As a practical matter you will never reach the entire original sample, so after a number of tries (typically 8-10 at a good firm), you substitute in another phone number from the same demographic profile to reach your sample size which gets you fairly close to a true random sample.

Many of the firms that try to do one day polls are especially vulnerable to this problem as in a "one day" you obviously have little chance to do callbacks.

Rasmussen (see note), and ARG are particularly vulnerable (IMHO) to this summer sampling effect.  I would be inclined to burn any summer poll from either of these two during the summer.

A second effect is in the "likely voter" models.

How the various firms sort out "likely voters" from the rest of the population varies a great deal from firm to firm, so it is hard to make any broad statements in this area, however firms which ask a lot of questions about "are you paying a lot of attention to the race" (or similar questions) will also tend to under represent Republicans during the summer.

Republicans tend to be older, more likely to be married, more likely to have kids, more likely to travel, and more likely to go on vacation than Democrats.

Consequently, during the summer this makes Republicans "tune out" a bit more than Democrats as they plan vacations, have the kids come home from college, get the kids ready for college, buy and sell homes, do yard work, head to the weekend cottage, etc…  Because of this, Republicans are less likely to be deemed "likely voters" in some polls, hence the GOP under polls a bit.

Among the firms that actually know what they are doing, Harris, Zogby and Gallup will tend to very modestly under poll Republicans a bit during the summer due to this effect – likely a couple points – no where near as bad as a lot of the University and local polls will.

Teeter/Hart (WSJ), TIPP, Democracy Corps, and Mason-Dixon should do fine in the summer as they base their "likely voter" more on past voter behavior than current attitudes.  Research 2000 cuts a lot of corners, but they at least are likely not to get any worse in the summer.

Fox uses registered voters in the summer so they are ok too.

Survey USA and ABC News use stratified samples so they should still work ok.

ARG, Newsweek, and CBS will all go even further to hell than normal in the summer.

****************************************************************

NOTE ON RASMUSSEN - Rasmussen uses something called a "cluster" sample (as does Survey USA) which in theory allows them to do an accurate one day poll.  The use of a "cluster sample" is a technique that I think must be described as experimental - it is unproven at best.

Rasmussen uses a "hard weight" by party ID in his national tracking poll, but does not use this weight in his state polls, so the comments I have made only directly apply to his state polls.

Survey USA samples differently than Rasmussen (something called "constructive sampling" or "sample stratification" as it is sometimes called) so SUSA should actually work fairly well in the summer.

[Sam Spade]

Another post on summer polls, from May 28, 2004

Firstly, burn all the state polls that come out in the summer.  Don't bother even looking at them.  Use them to wrap fish or line your bird cage.  I know the odd Dem will accuse me of bias on this but I have been saying this for months....

in the summer, you should take the state polls and burn them..

This leaves the national polls, so lets look at them...

(Since most polls are going registered voters, that is what I will use, in a "head to head" where possible.)

Firstly, the Head to Head Registered is the LEAST favorable to Bush of all the choices...

Gallup has Kerry +2
TIPP has Kerry +1
Fox has it tied
Quinnipiac has it tied
ABC has Kerry +2
Rasmussen (based on today's coin flip) has it tied
CBS News  has Kerry +47, or + 23 or what ever (ok it's 8% head to head)
CNN/Time which had Kerry +5%
Zogby has Kerry +5
ARG had Kerry +5

Even if we leave in CBS and ARG, if you average them all, you get Kerry up 2.8% nationally using the LEAST favorable for Bush way of looking at things there is. 

Knock 0.8% off for Nader, and Kerry is up 2% (A "sane" guess I think for a Nader adjustment factor)

Next, Remember this is registered voters we are talking here -   Give the GOP base (Which while grumpy is still showing 90% Support for Bush) anything RESEMBLING good news and the traditional 3% or so GOP edge in a likely voter versus registered vote poll pops back..

Like Magic, Bush is up a point or two...

There is also some evidence that Bush is running a tad better in the Battleground states than he is nationally

    -The last Gallup national poll had Bush +5 in the Battlegrounds
    -Fox had Bush +4 in the Battlegrounds
    -TIPP had Bush +6 in the battlegrounds

While the sample sizes in the battleground part of these national polls is quite small (240-360 ish) the fact that ALL THREE say roughly the same thing  makes me marginally more likely to believe it is true.

Some perspective here - Even without tossing out the clearly Bull%^&t polls in the above calculation, Bush is Down 2.8%, hardly a meltdown.

Ohio was carried by Bush by 3% and a bit of change in 2000 in a race he lost by .51% nationally.  Shift the whole nation 1.5% from it's 2000 baseline and Bush is up 1 or 2% or so in Ohio.

The media frenzy just reflect their enthusiasm for a good story.. the actual shift in the polls has been about 4% or so.  Bush was up 2% or so, he is now down 2% or so.

The fundementals of the race have not changed.

PREDICTION - Bush will be up 2 or 3 in the next round of National polls.

At least that is my opinion.

[○∙◄☻¥tπ[╪AV┼cVê└]

Is The Vorlon your kingpin?

[WMS]

Is The Vorlon your kingpin?

The Vorlon, all ideological differences aside, is a consummate professional in the realm of polling...like it or not.