Were states like Oregon and New Jersey on one side (or Virginia and Colorado on the other) viewed as legitimately competitive during the '04 campaign?
They were never considered top tier swing states, but there were moments in the final couple months where it looked like Bush was headed to a sizable victory (the sizable part obviously never materialized), and there was some speculation/uncertainty about these two states. Remember Oregon was extremely close in 2000. As for New Jersey, I recall the speculation about that state being in play for Bush was all in relation to NJ's satisfaction in Bush's response to 9/11.
And while polarization was very much a thing in the lead up to this election, the red state / blue state polarization polarization polarization world we live in wasn't really locked in until after this election, which resulted in a nearly identical, closely divided map as 4 years prior.