GOP Summer polling
The Vorlon:
-----Original Message-----
Subject: A question about polls during the summer.
Hey Vorlon,
This is Wildcard from the Atlas forums and I had a question for ya.
Is it true that the Republican base in polls normally gets weaker during the summer? I've heard this a few times and I was wondering if it's true or not.
Thanks man and have a good one!
Standard Summer State Poll warning.. again....
If a summer state poll does not say "Mason Dixon" - you should burn it... :)
Here is a quick reply I am “recycling” – hope it answers your question…
<<cut and paste from another email>>
A real quickie answer…
The “short” answer is, generally speaking, "yes"
Typically the GOP will under poll by 3-5% in the summer. (Democrats under poll by about 3% in late September/early October BTW)
The degree to which this is true depends heavily however on the quality of the firm doing the poll.
To a lesser degree it also depends on the nature of the voter screen used.
As a very broad generalization, the worse the poll, the more the GOP will under poll, the better the firm the less the GOP will under poll. – with a few exceptions I have noted below.
The "correct" answer is a tad more complicated.
A little bit of background.
Many of the lesser quality polling firms use "random digit dialing", (a fancy way of saying that they have a computer randomly pick phone numbers to call) to select who they talk to for a poll.
This is a flawed way to do a poll anyway unless you use something called "sample stratification" as per Zogby, ABC, SUSA etc (a whole other topic which I will leave for another day), but presents extra problems in the summer because Republicans tend to be on vacation, out of town, away for the weekend, etc a bit more than Democrats. - Obviously not being home, they are contacted less often by a pollster, which skews the sample….
A "good" firm will compensate for this with "call backs" but often the polling firms used by media outlets are selected because they are cheap, rather than because they are good.
Here is an extreme example of this effect.
Assume for a moment that the City of Green Bay (Population 250,000 or so?) had a ballot initiative to build the Packers a new football stadium, you were conducting a poll on this initiative, and that for some reason you decided to make all your calls while the Green Bay Packers were actually playing a home game ( ie 60,000 Packer fans were at Lambeau Field and thus not at home….)
Obviously the sample of people you would be able to actually reach would be very different than the actual population you were trying to sample. 60,000 Packer fans are at the game, and hence you can’t poll them… - as consequence your poll would likely be rather badly wrong.
On way to compensate for this is to do "call backs"
Rather than just keep dialing till you got your designated sample size, you would try everybody on your list once, and then stop.
The next day you would try again the people you could not reach on day one, and keep retrying on different days and at different times till you reached you entire original sample.
This way, a representative slice of Packer fans that were at the game would be included in your sample when you tried them on Monday, Tuesday, Wednesday….
As a practical matter you will never reach the entire original sample, so after a number of tries (typically 8-10 at a good firm), you substitute in another phone number from the same demographic profile to reach your sample size which gets you fairly close to a true random sample.
Many of the firms that try to do one day polls are especially vulnerable to this problem as in a "one day" you obviously have little chance to do callbacks.
Rasmussen (see note), and ARG are particularly vulnerable (IMHO) to this summer sampling effect. I would be inclined to burn any summer poll from either of these two during the summer.
A second effect is in the "likely voter" models.
How the various firms sort out "likely voters" from the rest of the population varies a great deal from firm to firm, so it is hard to make any broad statements in this area, however firms which ask a lot of questions about "are you paying a lot of attention to the race" (or similar questions) will also tend to under represent Republicans during the summer.
Republicans tend to be older, more likely to be married, more likely to have kids, more likely to travel, and more likely to go on vacation than Democrats.
Consequently, during the summer this makes Republicans "tune out" a bit more than Democrats as they plan vacations, have the kids come home from college, get the kids ready for college, buy and sell homes, do yard work, head to the weekend cottage, etc… Because of this, Republicans are less likely to be deemed "likely voters" in some polls, hence the GOP under polls a bit.
Among the firms that actually know what they are doing, Harris, Zogby and Gallup will tend to very modestly under poll Republicans a bit during the summer due to this effect – likely a couple points – no where near as bad as a lot of the University and local polls will.
Teeter/Hart (WSJ), TIPP, Democracy Corps, and Mason-Dixon should do fine in the summer as they base their "likely voter" more on past voter behavior than current attitudes. Research 2000 cuts a lot of corners, but they at least are likely not to get any worse in the summer.
Fox uses registered voters in the summer so they are ok too.
Survey USA and ABC News use stratified samples so they should still work ok.
ARG, Newsweek, and CBS will all go even further to hell than normal in the summer.
****************************************************************
NOTE ON RASMUSSEN - Rasmussen uses something called a "cluster" sample (as does Survey USA) which in theory allows them to do an accurate one day poll. The use of a "cluster sample" is a technique that I think must be described as experimental - it is unproven at best.
Rasmussen uses a "hard weight" by party ID in his national tracking poll, but does not use this weight in his state polls, so the comments I have made only directly apply to his state polls.
Survey USA samples differently than Rasmussen (something called "constructive sampling" or "sample stratification" as it is sometimes called) so SUSA should actually work fairly well in the summer.
JohnFKennedy:
Nice work there Vorlon.
I would argue that there is no such thing as a truly random sample though.
For those who don't know the definitions of the methods of sampling listed by The Vorlon, they are as follows:
Systematic Sample - A systematic sample is one that is random but where the choosing is based on some pre-described system.
e.g. Numbering each member of the population according to some criterion such as age/height and picking out every fifth one.
Stratified Sample - A stratified sample involves dividing the population into groups or strata. For each group you choose a random or systematic sample which reflects the proportion of that group within the population.
e.g. If the ratio of boys to girls is 2:1 in the population then the sample should reflect this.
I will be sure to burn all summer state polls that do not say Mason Dixon on them :).
The Vorlon:
Quote from: John F. Kennedy on May 24, 2004, 12:10:08 PM
I would argue that there is no such thing as a truly random sample though.
Agreed.
I believe my exact words were something like "fairly close to a true random sample"
The big problem with pure random digit dialing is you get a random sample of telephone numbers which is NOT the same thing as a random sample of voters
For example, a single peson tends to have 1 phone line (1 voter, 1 phone line) while a married couple also tends to have 1 phone line (2 voters - 1 phone line) hence in pure random digit dialing a single person has twice the chance of being contacted as a married person.... hence the sample is not random with respect to actual voters...
There is actually a company called (creatively enough) Survey Sample International which sells to the better pollsters (including Gallup) corrected telephone lists that have (among many other things) adjusted the frequency of telephone numbers within an exchange in proportion to the actual density of adult population per phone line, so the phone lines connected to a married couple (two votes) on average have twice the chance of being called as a phone belonging to a single person (one vote)
(It is waaaaaay more complicated than this, but it gives you the idea of what they do)
But that, as i said, is a whole different, and painfully complicated, discussion. :(
Don't even think about mentioning cell phones, which by law pollsters are not allowed to call, or I will have to hit you...
Cell phones represent to pollsters a problem for which there actually may not be a solution if the law stays in place.
Currently about 3% of the adult population has a cell phone as their only phone line.
As this % grows things get... complicated....
chop chop! :)
JohnFKennedy:
Quote from: The Vorlon (On Vacation till May 31) on May 24, 2004, 01:50:45 PM
Quote from: John F. Kennedy on May 24, 2004, 12:10:08 PM
I would argue that there is no such thing as a truly random sample though.
Agreed.
I believe my exact words were something like "fairly close to a true random sample"
chop chop! :)
heh, you wouldn't by chance happen to be an Actuary or a Statistician Vorlon?
BTW, I added you on msn.
The Vorlon:
Quote from: John F. Kennedy on May 24, 2004, 01:52:54 PM
Quote from: The Vorlon (On Vacation till May 31) on May 24, 2004, 01:50:45 PM
Quote from: John F. Kennedy on May 24, 2004, 12:10:08 PM
I would argue that there is no such thing as a truly random sample though.
Agreed.
I believe my exact words were something like "fairly close to a true random sample"
chop chop! :)
heh, you wouldn't by chance happen to be an Actuary or a Statistician Vorlon?
BTW, I added you on msn.
Good guess..... :)
M'Sc in Math back in 1988 - just finished up a month ago a quickie B'Sc in computer science to refresh to old brain and get it up to date. (My abacus from 1988 was a bit worn out)
See you on MSN!
:)
Navigation
[0] Message Index
[#] Next page