I believe the Romney campaign's confidence and the source of the "skewed polls" was because they predicted Democratic turnout would fall significantly as compared to 2008.
IIRC, Democrats were 39% of the electorate in 2008, and were predicted to fall to 34-36% in the polls favoring Romney, but in reality Democrats held up at 38% of the electorate.
It's not entirely an unreasonable belief, but as the polls kept showing high Democrat turnout as the election approached, it was a little naiive of them to not take Obama's ground game (which, whether you love him or hate him, is effing amazing) seriously.
If the consensus of all pollsters was that Democratic turnout was going to look a lot like 2008, why would believing otherwise be anything but unreasonable?