Is 2004 the best example of the "Keys to the White House" are junk?

[I spent the winter writing songs about getting better]

Indeed. Now let me explain. Here were the keys:

Midterm gains: After the midterm elections, the incumbent party holds more seats in the U.S. House of Representatives than after the previous midterm elections.
This is a rare time when this key is TRUE.

No primary contest: There is no serious contest for the incumbent party nomination.
Absolutely TRUE.

Incumbent seeking re-election: The incumbent party candidate is the sitting president.
Obviously TRUE.

No third party: There is no significant third party or independent campaign.
Also definitely TRUE.

Strong short-term economy: The economy is not in recession during the election campaign.
The economy wasn't great, but this would be TRUE.

Strong long-term economy: Real per capita economic growth during the term equals or exceeds mean growth during the previous two terms.
Without looking it up I'll assume this is FALSE, considering how strong it was during both Clinton terms and that it was shaky in 2004.

Major policy change: The incumbent administration effects major changes in national policy.
Yep, this has to count as TRUE.

No social unrest: There is no sustained social unrest during the term.
2020 was the first time this key was turned since 1992, definitely TRUE.

No scandal: The incumbent administration is untainted by major scandal.
TRUE, I know that most ardent liberals would argue otherwise but the high bar for "major" makes it tough to turn otherwise.

No foreign/military failure: The incumbent administration suffers no major failure in foreign or military affairs.
In 2004 this would probably be TRUE, although Lichtman did state that the Iraq War definitely qualified as a failure in 2008 (rightfully so.)

Major foreign/military success: The incumbent administration achieves a major success in foreign or military affairs.
Again this is a pretty subjective key, but either capturing Saddam or ousting the Taliban (ironic as that seems in hindsight) would probably be enough to count this as TRUE.

Charismatic incumbent: The incumbent party candidate is charismatic or a national hero.
FALSE, would not apply to Bush.

Uncharismatic challenger: The challenging party candidate is not charismatic or a national hero.
TRUE, absolutely.

So that's a minimum of two "false" keys, and you can quibble about some of the others but it would be pretty hackish to put the total number above three. Which is probably one of the strongest incumbent administrations in a century or so. So the test was correct, Bush won the popular vote right?

Well he did....but 2.5 points is a bit underwhelming and the most underwhelming popular vote victory aside from 2000 since 1976. And in fact the weakest popular vote victory of an incumbent party since the current party system was established....and actually probably ever (at least since you could have a meaningful count of the popular vote), and even the weakest popular vote victory for an incumbent party since 1880. So yes it's right, but there should be more correlation to the numbers than what actually happened. Like I won't go through 1984 or 1972 or even 1964, but I bet they had a weaker state for the incumbent party than 2004.

Now if Kerry had done 2.5 points better I'm sure Lichtman would fudge the keys and argue the status of Iraq counted as a failure and deny any successes, but even that just counts as four "false" keys with two more needed, maybe try to argue the scandals were "major" after all or that the tax cuts/Medicare Part D/the post-9/11 security overhaul weren't "major changes" or that Kerry's Vietnam War service and activism constituted him as a "national hero", point is the keys are easy to fudge and there's not a whole lot of correlation to the numbers.

[Alben Barkley]

Considering Bush that year is the ONLY Republican to win the popular vote at all since 1988, I’d say it’s not too bad.

Besides, the keys don’t purport to pick the size of a victory, just who wins.

[Independents against George Santos]

Bush was a weak incumbent in a favorable environment. Somebody else would have capitalized better, much like how somebody other than Trump would have done better in 2020.

[SWE]

It's 2016 that best debunks the validity of the keys.

The incumbency, party mandate, and primary challenge all have objective definitions, and all of them flipped against Hillary. Less objective but still fairly easy is the incumbent charisma key, which I don't think anyone would seriously argue Hillary had. Similarly, the policy change key was pretty clearly lost. Remember, the keys only look at the previous term, and not the entire administration, so Obamacare, a first term accomplishment, doesn't count. Pretty much all of Obama's second term policy goals were stonewalled by Congress, so this key is lost. Finally, you have the foreign policy victory key. The best argument for this would be the Cuban and Iran deals, which, while obviously good things, were too divisive and, from the perspective of the average persons priorities, not really important enough to compare to something like the death of Bin Laden or the surrender of the Japanese in WWII. That's six keys pretty clearly turned against the administration, which would indicate a Trump popular vote victory. I would also argue that emailgate, as stupid as it was, was taken seriously by enough Americans that the scandal key should have been flipped, but at that point it's just insult to injury, Hillary should be DOA as far as the PV is considered before you even get there.

The classic argument is that Trump won the EV and the keys are ambiguous as to whether or not the keys predict the PV or EV. Except no, they're not ambiguous. When Lichtman developed this theory, well before even 2000, he was always extremely explicit that the keys only predict the PV. When applied retroactively, the keys also indicate a Tilden victory over Hayes and a Cleveland victory over Harrison. After all, what else could they predict? The theory behind the keys is that the state of the country decides how most people will vote, nothing in the keys could possibly predict how the actual votes are geographically distributed, that doesn't make any sense at all.

Don't take my word for it, this is an excerpt from the edition of his book in which he predicts Trump's victory:

The keys to the White House focus on national concerns such as economic performance, policy initiatives, social unrest, presidential scandal, and successes and failures in foreign affairs. Thus, they predict only the national popular vote and not the vote within individual states. Indeed, no system could have predicted the 537 vote margin for George W. Bush in Florida that decided the 2000 election. In three elections since 1860, where the popular vote diverged from the electoral college tally—1876 (when Democrat Samuel J. Tilden won the popular vote, but lost in the electoral college to Republican Rutherford B. Hayes), 1888, and 2000—the keys accurately predicted the popular vote winner. Based on the historical odds since 1860, the chances are better than twelve to one that the popular and electoral college vote will converge in any given election. However, these odds presume continuity over time in the relationship between popular and electoral college votes. Some analysts have suggested, however, that this relationship may have changed given the sharp division in America between Republican ‘‘red states’’ and Democratic ‘‘blue states.’’

-Predicting the Next President: The Keys to the White House, 2016 Edition, Allan J. Lichtman, page xi

The same exact paragraph is kept in the 2020 edition, there's no way to argue that Lichtman's method correctly predicted the 2016 election

[I spent the winter writing songs about getting better]

It's 2016 that best debunks the validity of the keys.

The incumbency, party mandate, and primary challenge all have objective definitions, and all of them flipped against Hillary. Less objective but still fairly easy is the incumbent charisma key, which I don't think anyone would seriously argue Hillary had. Similarly, the policy change key was pretty clearly lost. Remember, the keys only look at the previous term, and not the entire administration, so Obamacare, a first term accomplishment, doesn't count. Pretty much all of Obama's second term policy goals were stonewalled by Congress, so this key is lost. Finally, you have the foreign policy victory key. The best argument for this would be the Cuban and Iran deals, which, while obviously good things, were too divisive and, from the perspective of the average persons priorities, not really important enough to compare to something like the death of Bin Laden or the surrender of the Japanese in WWII. That's six keys pretty clearly turned against the administration, which would indicate a Trump popular vote victory. I would also argue that emailgate, as stupid as it was, was taken seriously by enough Americans that the scandal key should have been flipped, but at that point it's just insult to injury, Hillary should be DOA as far as the PV is considered before you even get there.

The classic argument is that Trump won the EV and the keys are ambiguous as to whether or not the keys predict the PV or EV. Except no, they're not ambiguous. When Lichtman developed this theory, well before even 2000, he was always extremely explicit that the keys only predict the PV. When applied retroactively, the keys also indicate a Tilden victory over Hayes and a Cleveland victory over Harrison. After all, what else could they predict? The theory behind the keys is that the state of the country decides how most people will vote, nothing in the keys could possibly predict how the actual votes are geographically distributed, that doesn't make any sense at all.

Don't take my word for it, this is an excerpt from the edition of his book in which he predicts Trump's victory:

The keys to the White House focus on national concerns such as economic performance, policy initiatives, social unrest, presidential scandal, and successes and failures in foreign affairs. Thus, they predict only the national popular vote and not the vote within individual states. Indeed, no system could have predicted the 537 vote margin for George W. Bush in Florida that decided the 2000 election. In three elections since 1860, where the popular vote diverged from the electoral college tally—1876 (when Democrat Samuel J. Tilden won the popular vote, but lost in the electoral college to Republican Rutherford B. Hayes), 1888, and 2000—the keys accurately predicted the popular vote winner. Based on the historical odds since 1860, the chances are better than twelve to one that the popular and electoral college vote will converge in any given election. However, these odds presume continuity over time in the relationship between popular and electoral college votes. Some analysts have suggested, however, that this relationship may have changed given the sharp division in America between Republican ‘‘red states’’ and Democratic ‘‘blue states.’’

-Predicting the Next President: The Keys to the White House, 2016 Edition, Allan J. Lichtman, page xi

The same exact paragraph is kept in the 2020 edition, there's no way to argue that Lichtman's method correctly predicted the 2016 election

That's true but I remember Lichtman trying to worm his way into arguing that wasn't the case by claiming Sanders didn't count as a "major" challenge for some laughable reason. At some point before the election he even came up with the even more laughable take that the challenger charisma key turned "double" against Trump, as in Trump was so disliked that key counted against him as badly as two keys did. Kind of laughable.

[dw93]

2000 and 2016 are way better examples.

[Del Tachi]

You don't need "examples" to disprove the Lichtman test.  Anyone with common sense could tell you it's pseudoscientific garbage coupled with in-sample overfitting. 

The "keys" are so weasely and non-specific that you could make the test say anything you wanted.  Only with the benefit of hindsight does it become obvious that the test "worked."

[TheElectoralBoobyPrize]

Your analysis of the 2004 election is actually too favorable to Bush. He also lost the foreign-failure key (because of 9/11) and the policy change key (though I remember Lichtman saying it was a close call). He was down four keys. If he hadn't run, Republicans lose incumbency and likely nomination contest thus the Democrat wins.

A better election for making your case would be 1996. Clinton was down five keys...one away from defeat, but he won by eight points. Maybe Lichtman made a wrong call on a key...

Party Mandate: well obviously, this one was gone
3rd Party: Perot got more than 5%...albeit not by much
Policy Change: Because Clinton didn't completely reverse Reaganomics and stuck to centrist legislation
Foreign Policy Success: this would have been a hard key to get in the post-Cold War, pre-9/11 world
Incumbent Charisma: Clinton was a gifted speaker, but Lichtman thought there still too many doubts about his character even if those doubts were insufficient at the time to turn the scandal key

So yeah hard to believe that just one more setback and Dole would've won.

That's true but I remember Lichtman trying to worm his way into arguing that wasn't the case by claiming Sanders didn't count as a "major" challenge for some laughable reason. At some point before the election he even came up with the even more laughable take that the challenger charisma key turned "double" against Trump, as in Trump was so disliked that key counted against him as badly as two keys did. Kind of laughable.

Actually, I think the first argument is worse. He clearly states in the book that the nominee should get two-thirds of the delegates on the first ballot. Hillary got three-fifths of the delegates on the first ballot. Does Lichtman need remedial math?

[OSR stands with Israel]

2000 and 2016 are way better examples.

Lichtman predicted Trump would win in 16

[dw93]

2000 and 2016 are way better examples.

Lichtman predicted Trump would win in 16

You're right he did, but he said when he predicted Gore's win in 2000 and Gore ended up losing to Bush that his model was based on the popular vote, which Gore won. Then in 2016, he said Trump would win but Trump lost the popular vote. My point in my original post was that Lichtman contracted himself with those election, thus the model is a bit junky.

[Mr. Smith]

The Redskins Rule and Halloween Masks ones were funnier.

But yeah, 2016 and 2000 are better examples here.

[Alben Barkley]

You don't need "examples" to disprove the Lichtman test.  Anyone with common sense could tell you it's pseudoscientific garbage coupled with in-sample overfitting. 

The "keys" are so weasely and non-specific that you could make the test say anything you wanted.  Only with the benefit of hindsight does it become obvious that the test "worked."

Whatever the value of the model itself, his predictions (made every time before the election, so not just retrofitted after, and based on a nearly 40 year old model he doesn’t change), are still a lot more accurate than chance. Or most other people’s predictions for that matter. I’d argue that since Gore actually won in 2000 in both the popular vote and electoral college, he’s actually never been wrong.

I used to call him a charlatan and never thought I’d find myself in the position of semi-defending him, but frankly his predictions have proven no worse if not better than polling-based predictions in recent years.

[○∙◄☻¥tπ[╪AV┼cVê└]

Here's 1856.

Midterm gains: After the midterm elections, the incumbent party holds more seats in the U.S. House of Representatives than after the previous midterm elections.
Lost 69 seats, FALSE

No primary contest: There is no serious contest for the incumbent party nomination.
Incumbent defeated and wasn't even one of the last 2 candidates, FALSE

Incumbent seeking re-election: The incumbent party candidate is the sitting president.
Incumbent defeated in primary, FALSE

No third party: There is no significant third party or independent campaign.
3rd place was 21%, FALSE

Strong short-term economy: The economy is not in recession during the election campaign.
TRUE

Strong long-term economy: Real per capita economic growth during the term equals or exceeds mean growth during the previous two terms.
There was a 1853-1854 recession and no recession in the previous term, so PROBABLY FALSE

Major policy change: The incumbent administration effects major changes in national policy.
There was the Kansas-Nebraska Act, but oof. TRUE

No social unrest: There is no sustained social unrest during the term.
Bleeding Kansas, FALSE

No scandal: The incumbent administration is untainted by major scandal.
The Kansas-Nebraska Act was considered a betrayal, so UNSURE

No foreign/military failure: The incumbent administration suffers no major failure in foreign or military affairs.
The attempt to annex Cuba backfired, FALSE

Major foreign/military success: The incumbent administration achieves a major success in foreign or military affairs.
I guess the Gadsden Purchase., TRUE


Charismatic incumbent: The incumbent party candidate is charismatic or a national hero.
I don't think so, FALSE

Uncharismatic challenger: The challenging party candidate is not charismatic or a national hero.
Fremont led the Bear Flag Republic, FALSE

[OSR stands with Israel]

You don't need "examples" to disprove the Lichtman test.  Anyone with common sense could tell you it's pseudoscientific garbage coupled with in-sample overfitting. 

The "keys" are so weasely and non-specific that you could make the test say anything you wanted.  Only with the benefit of hindsight does it become obvious that the test "worked."

Whatever the value of the model itself, his predictions (made every time before the election, so not just retrofitted after, and based on a nearly 40 year old model he doesn’t change), are still a lot more accurate than chance. Or most other people’s predictions for that matter. I’d argue that since Gore actually won in 2000 in both the popular vote and electoral college, he’s actually never been wrong.

I used to call him a charlatan and never thought I’d find myself in the position of semi-defending him, but frankly his predictions have proven no worse if not better than polling-based predictions in recent years.

Lichtmann Keys really just go off the fact that all that matters in elections are the fundamentals along with some candidate charisma added in as a bonus. That isnt really isnt even wrong cause even just by taking a glance at the fundamentals the only elections you can really argue went against the fundamentals were 1960 and 2000.

Adding in charisma , 2000 becomes the only outlier which is the only election Lichtmann got wrong and 2000 is one where the Democratic Candidate ran away from the fundamentals that would have said the Democrats should have won that election

[Lurker]

You don't need "examples" to disprove the Lichtman test.  Anyone with common sense could tell you it's pseudoscientific garbage coupled with in-sample overfitting. 

The "keys" are so weasely and non-specific that you could make the test say anything you wanted.  Only with the benefit of hindsight does it become obvious that the test "worked."

Whatever the value of the model itself, his predictions (made every time before the election, so not just retrofitted after, and based on a nearly 40 year old model he doesn’t change), are still a lot more accurate than chance. Or most other people’s predictions for that matter. I’d argue that since Gore actually won in 2000 in both the popular vote and electoral college, he’s actually never been wrong.

I used to call him a charlatan and never thought I’d find myself in the position of semi-defending him, but frankly his predictions have proven no worse if not better than polling-based predictions in recent years.

How long before the election is this?