So, what then is the value of forecasting, if it can never be proven right or wrong? That is, if we're saying there's an 80% chance of Republican control of the Senate on election day, but the Democrats wind up winning, the forecaster can just say, "This was part of the 20%." So what was the value of the exercise?
Over time, when he says something has an 80% chance, it should happen around 80% of the time and not happen 20% of the time. With a large enough sample size, we can test to see if his model is good. If the things he has have an 80% probability only happen 60% of the time, his model sucks. Similarly, if they happen 90% of the time, his model sucks.
Remember, he's supposed to be "wrong" 20% of the time in this scenario. If he's only "wrong" 5% of the time, his model is awful, even if he probably wouldn't get the discredit.
Since we'll never get a large enough sample size of his outcomes, the model is worthless and should be thrown out. Silver hasn't been all that impressive with his outcomes overall. Throw it on the pile or in the trash.
