If Nate Silver had to analyse Nate Silver's predictions with an unbiased eye, what he'd likely say is there's too small a sample size to get a good idea of exactly how his models are biased. It's clear he hasn't predicted 95%+ or whatever of races by chance alone, but if there's a pattern in his misses? People are making a huge deal over 2 close Senate races in the same year. I wouldn't go so far as to say that it means nothing, perhaps there is something to it, but there isn't enough data to say so with much confidence.
I would argue the biggest flaw in Silver's system is that it overestimates the likelihood of outcomes considered "Safe", especially early in the cycle. For instance, look at the earlier columns from the 2008 chart Harry posted. Notice the plethora of seats with really lopsided percentages (NC, OR, MN) that changed drastically as the race unfolded. That seems like too much variation to accept, say that as of July 20th the Democrats truly only had a 9% chance of taking the NC senate seat.
Of course, the model can't be much better than the polling it has to work with. I also find the margin of error reported by polling companies to be particularly useless because the reported margin is how far off we would expect the poll to be (at a given confidence level) only due to statistical variation. That assumes the poll was a sample that is actually representative of the electorate. Therein lies the greater problem.
