Applying a swingometer to US Elections

[politicaltipster]

In the UK, because there are over 600 individual seats, pollsters apply a Uniform National Swing to past results. For instance, if national polls suggest that the Labour lead over the Conservatives will be 3% less than last time they simply assume that there will be a uniform national swing of 3% in each seat.  Conventional wisdom suggests that you can't do this with US Elections because, even at the national level, 'all politics is local'. However, applying a UNS to the 2004 election is surprisingly accurate.

http://thepoliticaltipster.wordpress.com/2007/11/17/can-we-use-the-swingometer-to-predict-us-elections/

Has anyone tried this with congressional elections? What do you think of the idea?

[Erc]

Generally, it will work decently for races involving the same President (look at the similarity between '92 & '96, or '00 & '04), but will have serious issues when there are third party candidates of any significance (Nader included) or when the candidates are different.

2000 election under uniform swing from '96:

Actually, doesn't perform that horribly, but fails spectacularly in a couple of areas.  AR & LA are given to the Democrats on the basis of Clinton's great performance in those states, which was due to the fact that he's from AR, that he performed better in the South than Gore, and that Perot underperformed in the South.

It also gets WV wrong, but that took a lot of other people by surprise, as well.

Other elections, with more dramatic shifts, it will completely screw up:

That's 1976 under a shift from 1972.  It horribly messes up the South, moderately messes up the West, gives SD to Carter based on McGovern's performance, and fails to account for Ford's strength in Michigan, among others.  Carter still wins, but that's more a statement on the fact that the PV is a pretty good indicator of an EV win, regardless, than anything else--the Swingometer assigned half (half!) of the states incorrectly.
[Interestingly, the Swingometer also puts 10 states within 2%]


There's nothing wrong with using a swingometer, as long as you know its limitations very well and adjust for certain obvious facts on the ground.  In the end, a careful following of polling data will do you just as well, as long as you avoid partisan bias meddling with your predictions.  I got 48/50 right in 2004...and if it hadn't been for a bit of unwarranted optimism on my part, I'd have put WI & NH in the Kerry column & gotten 50/50 right.

[Хahar 🤔]

I was going to try this, but my computer somehow shut down and I haven't had time to restart it.

[Mr. Morden]

What if you just don't care about which specific states each candidate wins, but are interested in using a swing-ometer just to estimate the number of electoral votes each candidate would win, given the popular vote breakdown?  How well would it work then?  Can you produce a very rough formula for likely range of electoral votes won, given a particular popular vote result?

Also, how well would it work for the House of Reps?  Again, if you don't care about which specific seats flip, but just want to estimate the raw number of seats switching for a particular shift in the "national popular vote" in the House?

[politicaltipster]

It would be interesting if someone could find the 2004 and 2006 congressional results in a format that didn't involve massive fiddling around. The FEC produce data for 1996-2004 in spreadsheet format but manage to lay it out so badly that it is little better than inputting the stuff manually.

[Erc]

The Clerk of the U.S. Congress has 2004 data in not-so-horrible format, that I was able to use...

From 2004 to 2006, there was a 5.5% swing towards the Democrats as a fraction of the two-party vote.

Applying a national uniform swing to each seat, 17 seats change hands, changing the composition of the US Congress from 232-202-1 (R-D-I) to 215-219-1.

In actuality, 30 seats changed hands, plus VT-AL switched from Independent to Democrat, making it 202-233-0.

The swingometer thus underestimated the number of seats that would switch hands.

The seats that fell, according to the Swingometer, in order of swing necessary to take the seat (* denotes the seat actually fell):

IN-9 * (Sodrel-Hill)
PA-6   (Gerlach-Murphy)
CT-4   (Shays-Farrell)
WA-8  (Reichert-Burner)
CO-4   (Musgrave-Paccione)
MN-6   (open seat: Bachman-Wetterling)
NY-26  (Kuhl-Massa)
CT-2 * (Simmons-Courtney)
NM-1   (Wilson-Madrid)
IN-8 * (Hostettler-Ellsworth)
IN-2 * (Chocola-Donnelly)
NC-11 * (Taylor-Shuler)
LA-7    (Boustany-Stagg)
VA-2   (open seat: Drake-Kellam)
TX-32  (Sessions-Pryor)
FL-13  (Buchanan-Jennings) [results quasi-disputed]
KY-4    (Davis-Lucas)

So the swingometer is pretty broken as a predictor of which seats will fall, as well--only 5 of the 17 predicted seats fell [those who'd faced tough campaigns last time either A) didn't face the same tough campaigns this time [due to different incumbents or incumbency], or B) having faced tough challenges before, were well-equipped to handle 2006 being a bad year (Wilson, Gerlach)]

The other 25 seats that fell, by order of required swing:
IA-1: 6.0%
PA-8: 6.1%
CO-7: 6.1%
TX-22: 7.3%
KS-2: 7.6%
PA-7: 9.3%
IA-2: 10.1%
NY-24: 10.1%
NH-2: 10.4%
AZ-5: 10.9%  (the median seat to fall.  Had all seats more 'swingometer-vulnerable' than this fallen, 58 (!) seats would have fallen to the Democrats)
CT-5: 11.0%
CA-11: 11.2%
KY-3: 11.4%
AZ-8: 12.5%
MN-1: 12.7%
NY-20: 12.8%
NY-19: 13.5%
PA-4: 13.7%
FL-22: 14.0%
OH-18: 16.2%
FL-16: 18.0%
WI-8: 20.2%
TX-23: 20.2%
PA-10: 50%  (Sherwood went unopposed in 2004)

(not to mention VT-AL, of course)

[minionofmidas]

re Erc - five Texas seats were redrawn by the court, including 23 (but none of the other TX seats mentioned), which would have moved well up the table as a result. Just a minor quibble though.

[Erc]

re Erc - five Texas seats were redrawn by the court, including 23 (but none of the other TX seats mentioned), which would have moved well up the table as a result. Just a minor quibble though.

Yeah...I thought the result for TX-23 was a bit weird.

The major redistricting in Texas was between '02 & '04, though, right?

[minionofmidas]

re Erc - five Texas seats were redrawn by the court, including 23 (but none of the other TX seats mentioned), which would have moved well up the table as a result. Just a minor quibble though.

Yeah...I thought the result for TX-23 was a bit weird.

The major redistricting in Texas was between '02 & '04, though, right?

Yes. Parts of the 04 regerrymandering (which affected all 32 seats, though some only marginally) were overturned by the High Court.

[I spent the winter writing songs about getting better]

The Clerk of the U.S. Congress has 2004 data in not-so-horrible format, that I was able to use...

From 2004 to 2006, there was a 5.5% swing towards the Democrats as a fraction of the two-party vote.

Applying a national uniform swing to each seat, 17 seats change hands, changing the composition of the US Congress from 232-202-1 (R-D-I) to 215-219-1.

In actuality, 30 seats changed hands, plus VT-AL switched from Independent to Democrat, making it 202-233-0.

The swingometer thus underestimated the number of seats that would switch hands.

The seats that fell, according to the Swingometer, in order of swing necessary to take the seat (* denotes the seat actually fell):

IN-9 * (Sodrel-Hill)
PA-6   (Gerlach-Murphy)
CT-4   (Shays-Farrell)
WA-8  (Reichert-Burner)
CO-4   (Musgrave-Paccione)
MN-6   (open seat: Bachman-Wetterling)
NY-26  (Kuhl-Massa)
CT-2 * (Simmons-Courtney)
NM-1   (Wilson-Madrid)
IN-8 * (Hostettler-Ellsworth)
IN-2 * (Chocola-Donnelly)
NC-11 * (Taylor-Shuler)
LA-7    (Boustany-Stagg)
VA-2   (open seat: Drake-Kellam)
TX-32  (Sessions-Pryor)
FL-13  (Buchanan-Jennings) [results quasi-disputed]
KY-4    (Davis-Lucas)

So the swingometer is pretty broken as a predictor of which seats will fall, as well--only 5 of the 17 predicted seats fell [those who'd faced tough campaigns last time either A) didn't face the same tough campaigns this time [due to different incumbents or incumbency], or B) having faced tough challenges before, were well-equipped to handle 2006 being a bad year (Wilson, Gerlach)]

The other 25 seats that fell, by order of required swing:
IA-1: 6.0%
PA-8: 6.1%
CO-7: 6.1%
TX-22: 7.3%
KS-2: 7.6%
PA-7: 9.3%
IA-2: 10.1%
NY-24: 10.1%
NH-2: 10.4%
AZ-5: 10.9%  (the median seat to fall.  Had all seats more 'swingometer-vulnerable' than this fallen, 58 (!) seats would have fallen to the Democrats)
CT-5: 11.0%
CA-11: 11.2%
KY-3: 11.4%
AZ-8: 12.5%
MN-1: 12.7%
NY-20: 12.8%
NY-19: 13.5%
PA-4: 13.7%
FL-22: 14.0%
OH-18: 16.2%
FL-16: 18.0%
WI-8: 20.2%
TX-23: 20.2%
PA-10: 50%  (Sherwood went unopposed in 2004)

(not to mention VT-AL, of course)

Of course comparing an open seat (as most of those lower ones were) to one with a strong incumbent is pointless.

[minionofmidas]

The Clerk of the U.S. Congress has 2004 data in not-so-horrible format, that I was able to use...

From 2004 to 2006, there was a 5.5% swing towards the Democrats as a fraction of the two-party vote.

Applying a national uniform swing to each seat, 17 seats change hands, changing the composition of the US Congress from 232-202-1 (R-D-I) to 215-219-1.

In actuality, 30 seats changed hands, plus VT-AL switched from Independent to Democrat, making it 202-233-0.

The swingometer thus underestimated the number of seats that would switch hands.

The seats that fell, according to the Swingometer, in order of swing necessary to take the seat (* denotes the seat actually fell):

IN-9 * (Sodrel-Hill)
PA-6   (Gerlach-Murphy)
CT-4   (Shays-Farrell)
WA-8  (Reichert-Burner)
CO-4   (Musgrave-Paccione)
MN-6   (open seat: Bachman-Wetterling)
NY-26  (Kuhl-Massa)
CT-2 * (Simmons-Courtney)
NM-1   (Wilson-Madrid)
IN-8 * (Hostettler-Ellsworth)
IN-2 * (Chocola-Donnelly)
NC-11 * (Taylor-Shuler)
LA-7    (Boustany-Stagg)
VA-2   (open seat: Drake-Kellam)
TX-32  (Sessions-Pryor)
FL-13  (Buchanan-Jennings) [results quasi-disputed]
KY-4    (Davis-Lucas)

So the swingometer is pretty broken as a predictor of which seats will fall, as well--only 5 of the 17 predicted seats fell [those who'd faced tough campaigns last time either A) didn't face the same tough campaigns this time [due to different incumbents or incumbency], or B) having faced tough challenges before, were well-equipped to handle 2006 being a bad year (Wilson, Gerlach)]

The other 25 seats that fell, by order of required swing:
IA-1: 6.0%
PA-8: 6.1%
CO-7: 6.1%
TX-22: 7.3%
KS-2: 7.6%
PA-7: 9.3%
IA-2: 10.1%
NY-24: 10.1%
NH-2: 10.4%
AZ-5: 10.9%  (the median seat to fall.  Had all seats more 'swingometer-vulnerable' than this fallen, 58 (!) seats would have fallen to the Democrats)
CT-5: 11.0%
CA-11: 11.2%
KY-3: 11.4%
AZ-8: 12.5%
MN-1: 12.7%
NY-20: 12.8%
NY-19: 13.5%
PA-4: 13.7%
FL-22: 14.0%
OH-18: 16.2%
FL-16: 18.0%
WI-8: 20.2%
TX-23: 20.2%
PA-10: 50%  (Sherwood went unopposed in 2004)

(not to mention VT-AL, of course)

Of course comparing an open seat (as most of those lower ones were) to one with a strong incumbent is pointless.

And because that, in the US, is a matter of course, swingometers are pointless in the US, and are therefore not used. Ie, proving that what you just said is true was the point of the exercise really.

[Erc]

If every midterm race gets as much press as the last one did, it's possible that they might get more useful in the future (if the Congressional race becomes 'nationalized' in the media)...but I don't think that's going to happen, especially with Presidential races every other election.

[Mr. Morden]

OK, so it works badly for House races.  What about presidential elections?  Would a swingometer give you a good estimate of the expected number of electoral votes each candidate gets, for a given popular vote result?  (Again, assuming you don't care about predicting the specific state, just the total electoral vote count.)  That is, can you produce a formula that gives you the approx. expected electoral vote count, given the popular vote?

[Erc]

Erc's Very Atrocious Presidential Swingometer:  (by request of Mr. Morden)

Looking at each election, and the swing (from 50-50) required to gain electoral votes, one finds that the relationship between popular vote and electoral votes is roughly linear (for non-landslide elections, obviously--it's rather logistic for the blowouts).  Roughly estimating this dependency (by simple linear regression on states no more than 10% from the national average) yields the following relationship for the (Republican) electoral vote.

EV-R = 270 + k * p + a

Where:
p is the percentage of the two-way popular vote the republicans receive in excess of 50 (which could be negative).
k measures the response of the electoral vote to the popular vote.  This (for 1960 on) ranges from a high of 39.01 for 1976, to a low of 22.02 for 2004.  This parameter has been monotonically decreasing since 1976, but was pretty stable between 2000 (22.93) and 2004.
a is the 'natural Republican advantage' in electoral votes (due to the small state effect).  Unlike estimates from the straight-up swing, this is pretty stable, interestingly--ranging from 1.8 in 1960 and 6.5 in 1980 to 19.6 in 1972.  Most of the time, however, it stays at around 10, with a value of 9.7 in 2004 and 10.0 in 2000.

The quotient  2 a / k gives a sense of how large the range of possible "PV win but EV loss" is.  Given 2004 parameters, 2 a / k is .88%--that is, the Republican can lose to the Democrat by up to .88% while still winning in the EV.  (For comparison, Bush lost to Gore by .51% in the PV)

General notes:  The sensitivity tends to be larger in very close races (1960 or 1976, for canonical examples)--so this method may predict a tighter EV race than is actually the case in close (< 2% margin) PV races.
It also will not work, for obvious reasons, in blowouts on the scale of 1964 & 1972.

For a self-consistency check, let's apply this method to each year (first with that year's parameters, then with the previous election's):  (1984, 1972, and 1964 are omitted as they predict negative EV totals for the loser--these are blowouts beyond the predictive range of the model)

2004:  (Predicted (current parameters) vs. Predicted (lagged parameters) vs. Actual)
307 - 231   309 - 229   286 - 251   (the chips broke against Bush in a couple of states, like Wisconsin)

2000:
274 - 264   277 - 261   271 - 267    (This predicts the Bush win)

1996:
157 - 381   137 - 401   159 - 379

1992:
175 - 363   173 - 365   168 - 370

1988:
399 - 139   408 - 130   426 - 111

1980:
449 - 89   489 - 49   489 - 49          (lagged prediction is spot-on [by sheer coincidence])

1976:
297 - 241   263 - 275   297 - 241  [exactly right (by sheer coincidence), but lagged prediction suffers from 'close race' syndrome]

1968:
308 - 230   288 - 250   301 - 191 - 46   [3rd party presence is reflected in a ridiculous 'Republican Advantage' term this year of 26 EV's]

1960:
269 - 268   270 - 267   303 - 220 - 14   [Only race it gets wrong, and only race it really messes up.  This is mainly a product of the 'sensitivity in close races' condition, which is a product, in turn, of the extreme non-polarization of the country in 1960--the 10% cuts may not be great here]

So it's a fun, if crude estimator--liable to be wrong by +/- 20, easily. 

As a cute prediction:  Let's try 2008 with a swing from 2004 comparable to the 2004 - 2006 swing (45.9% Republican two-way vote share), on 2004 parameters (k = 22, a = 10):

Erc's Very Atrocious Presidential Swingometer predicts a 349 - 149 EV win for the Democrats.   There's a not-too implausible scenario with this exact outcome (which was coincidentally my first prediction for the race):

[minionofmidas]

Your model seems to predict that Giuliani will lose the Republican nomination to Huckabee.

[politicaltipster]

I've looked at the GOP share of the two party vote from 1960 to 2006 inclusive and the number of seats and using the forecast function in excel and come up with the following rough equation:

829.669854x - 207.6627 where x is the share of the two party vote (50% = 0.5). Using the forecast function it calls the house correctly in 21 out of the 24 elections (including 3 out of 6 of the Republican victories). It suggests that the GOP need to repeat their 2004 share of the vote to win. Interestingly, it suggests that the GOP should have got 28 seats less than they did last November. The difference between the forecast and the actual results tends to revert to 0, so that an election where one party under or over-performs tends to move toward the predicted result - a classic example being the GOP gaining seats in 1996 despite their share of the 2 party vote falling.

It is very possible that the election for the House could be much closer in terms of the popular vote than '06 but that the Democrats still make some gains. Consequently I've sold some of contracts on a GOP victory.

[politicaltipster]

I've actually produced a simple model which is:


GOP Seats = 4.255x + 195.446

Where x = GOP% of national vote - DEM% of national vote

The model has a r (correlation) of 0.902 and an r2 of 0.81. It correctly predicts control of the House 21 out of 24 times since 1960 with the worst errors being 2006 (GOP getting 28 more seats than expected) and 1976 (GOP getting 18 less seats than expected).

[StateBoiler]

Does your swingometer account for races by removing them if one of the two major parties is not running? Including results of House districts where a Democrat gets 82% against a Libertarian (or 100% when there is no opposition) for example shouldn't really be used in a national swing meter.

[politicaltipster]

Fair enough. However, my idea is that uncontested seats should balance each other out on the national level and that in any case the seats in question would have been a cakewalk for either party. It should be noted that, as you can see above, I've pretty given up trying to use a traditional swingometer to forecast individual seats and gone for the pseudo-swingometer approach of trying to find a historical linear relationship between seats and vote margins.

[Harry Hayfield]

Erc's Very Atrocious Presidential Swingometer:  (by request of Mr. Morden)

Looking at each election, and the swing (from 50-50) required to gain electoral votes, one finds that the relationship between popular vote and electoral votes is roughly linear (for non-landslide elections, obviously--it's rather logistic for the blowouts).  Roughly estimating this dependency (by simple linear regression on states no more than 10% from the national average) yields the following relationship for the (Republican) electoral vote.

EV-R = 270 + k * p + a

Where:
p is the percentage of the two-way popular vote the republicans receive in excess of 50 (which could be negative).
k measures the response of the electoral vote to the popular vote.  This (for 1960 on) ranges from a high of 39.01 for 1976, to a low of 22.02 for 2004.  This parameter has been monotonically decreasing since 1976, but was pretty stable between 2000 (22.93) and 2004.
a is the 'natural Republican advantage' in electoral votes (due to the small state effect).  Unlike estimates from the straight-up swing, this is pretty stable, interestingly--ranging from 1.8 in 1960 and 6.5 in 1980 to 19.6 in 1972.  Most of the time, however, it stays at around 10, with a value of 9.7 in 2004 and 10.0 in 2000.

The quotient  2 a / k gives a sense of how large the range of possible "PV win but EV loss" is.  Given 2004 parameters, 2 a / k is .88%--that is, the Republican can lose to the Democrat by up to .88% while still winning in the EV.  (For comparison, Bush lost to Gore by .51% in the PV)

General notes:  The sensitivity tends to be larger in very close races (1960 or 1976, for canonical examples)--so this method may predict a tighter EV race than is actually the case in close (< 2% margin) PV races.
It also will not work, for obvious reasons, in blowouts on the scale of 1964 & 1972.

For a self-consistency check, let's apply this method to each year (first with that year's parameters, then with the previous election's):  (1984, 1972, and 1964 are omitted as they predict negative EV totals for the loser--these are blowouts beyond the predictive range of the model)

2004:  (Predicted (current parameters) vs. Predicted (lagged parameters) vs. Actual)
307 - 231   309 - 229   286 - 251   (the chips broke against Bush in a couple of states, like Wisconsin)

2000:
274 - 264   277 - 261   271 - 267    (This predicts the Bush win)

1996:
157 - 381   137 - 401   159 - 379

1992:
175 - 363   173 - 365   168 - 370

1988:
399 - 139   408 - 130   426 - 111

1980:
449 - 89   489 - 49   489 - 49          (lagged prediction is spot-on [by sheer coincidence])

1976:
297 - 241   263 - 275   297 - 241  [exactly right (by sheer coincidence), but lagged prediction suffers from 'close race' syndrome]

1968:
308 - 230   288 - 250   301 - 191 - 46   [3rd party presence is reflected in a ridiculous 'Republican Advantage' term this year of 26 EV's]

1960:
269 - 268   270 - 267   303 - 220 - 14   [Only race it gets wrong, and only race it really messes up.  This is mainly a product of the 'sensitivity in close races' condition, which is a product, in turn, of the extreme non-polarization of the country in 1960--the 10% cuts may not be great here]

So it's a fun, if crude estimator--liable to be wrong by +/- 20, easily. 

As a cute prediction:  Let's try 2008 with a swing from 2004 comparable to the 2004 - 2006 swing (45.9% Republican two-way vote share), on 2004 parameters (k = 22, a = 10):

Erc's Very Atrocious Presidential Swingometer predicts a 349 - 149 EV win for the Democrats.   There's a not-too implausible scenario with this exact outcome (which was coincidentally my first prediction for the race):

I remember a comment that the swing in NH at the primary election was 4.4% from GOP to Dem. When I entered that into my own UNS swingometer forecaster I came up with gives Dems 349 (gaining CO, FL, IA, MO, NV, NM, OH, and VA). I would like to know what the other swings in the primaries have been in order to generate a forecast based on the average swing in the primaries so far.