Van Buren without a doubt. Polk is absolutely unacceptable: most people don't realize just how underhanded his conduct in the Mexican-American War was. Taylor turned out to be a decent president with hindsight, but at the time I would have been suspicious of him due to his Southern roots and his seeming lack of a concrete ideology.
The main concern with Van Buren: How would the South have responded if he had been elected? Is it possible that his Free Soil platform could have pushed America into an early Civil War or similar sort of event? IMO, it's unlikely. Most of the South had voted for him in 1836, so he wasn't disliked in the same way that Lincoln was, and Southern politics wasn't as dominated by fire eaters in 1848 as in 1860. Best-case scenario with a Van Buren election is slavery being strangled early with no Civil War, and I think that possibility would have been worth voting for.
I disagree. The South had serious reservations about Van Buren even in 1836: without Jackson's support, I don't think he could have carried the region. Six slave states, including South Carolina, voted for the Whigs that year, and Virginia's electors refused to support Van Buren's running mate. Furthermore, Van Buren's 1848 platform was virtually identical to Lincoln's in 1860. In my opinion, the South would not have tolerated the election of an anti-slavery president, even one who had been their ally in the past.
Poll
