I'm sure you can find prominent slaveholders who remained loyal to the Union during the war, particularly of course in Kentucky, where the planter class strategically aligned themselves with the Lincoln administration in hopes that if the war did bring about the end of slavery, it wouldn't effect them. HOWEVER, these individuals do not present the "typical Southern unionist." If we look to the places in the slave states where unionism was strongest, we see a strong inverse correlation at the county level between the % of the population who were slaves (i.e. the prevalence of slavery in that county) and unionist sentiment. Viz.,
From these maps, we see the large majority of unionist delegates to state conventions in the winter and spring of 1861 came from counties where slaves accounted for a negligible minority of the population, and therefore where the white male electorate was far less likely to include major slaveowners. The areas where slavery was strongest, with some exceptions, sent delegates who supported the secession ordinances and many of whom in fact had been angling for this outcome for over a decade. So no, secession was emphatically not the pet project of dirt farmers in western North Carolina and eastern Tennessee: it was emphatically supported by the majority of the planter class, and its opponents were mostly very poor whites in the mold of Andrew Johnson who hated blacks, but hated the planters more.
A distinction needs to be drawn between those who in the 1850s opposed policies which inevitably would result in secession, and how they behaved after.
Fire Eater politics prior to November 1860 were flakish. They could lead to nothing could and much of the genuine elite opposed them.
Once South Carolina left the die was cast. Anyone still backing the Union after Fort Sumter was in effect weakening the leverage of the South either to succeed in secession or to force a settlement. It is ahistoric to treat those who backed the Confederacy once it was an established fact with their behavior before, especially because Johnson was far more radical in the 1850s than Bell. For instance on Kansas-Nebraska where Bell opposed it and Johnson supported it. Arguably the Johnson/Bell dynamic is exactly what I meant. Bell represented a fundamentally conservative constituency which opposed the Kansas-Nebraska act and favored accepting limitations on slavery in order to avoid conflict when populists like Johnson wished to fight abstract principled battles. In 1861, Bell again bowed to "reality" namely that with the Deep South gone the only options were for the Deep South to be crushed which would destroy the position of the Upper South, or for it to succeed. There was no longer much chance to avoid war, and what chance existed lay in the South sticking together.
In effect
1850s - Large slaveowners tended to be far less engaged in the fights over abstract "Southern Rights" because they stood to gain nothing economically in Kansas and to lose enormously if the conflict led to the triumph of "demagogues" in the North(and South). Hence they saw efforts to expand slavery as threats to their own position and the security of their own property.
1858-60 - They opposed the takeover of the Southern Democratic party by ideologies, almost none of whom owned extensive numbers of slaves, who wanted a breakup and floated fantasies such as reopening the slave trade or suggesting Dred Scott required slavery to be expanded nationally. They made up the base of residual Douglas support in the South and Bell support.
1861 - Your map is right but studies show that there is not a very good correlation between 1860 Presidential support and 1861 secession voting in the Deep South after the first few states. Whigs like Alexander Stephens led the fight against secession in Georgia, but once it was clear several states were going out, the former Whigs generally fell into line. The downscale Democrats did not, which is why they made up almost all 1861 Unionists.
But 1861 Unionism, and post 1861 Unionism is a very different phenomenon. At that point it is purely ideological. There is nothing anyone with property can hope to preserve or accomplish through Unionism. The only options that can save them are a Southern Victory or a compromise. Unconditional Unionism makes both less likely. A Northern political or military victory at that point is the end.
In effect large property owners behaved how one would expect them to behave. They sought to avoid abstract political clashes where they stood to gain nothing, and potentially lose much. When a binary situation developed they again eschewed seeking mythical third options and chose to make the best of a bad situation. Even if they believed the South could not win a war, that was immaterial. The war was happening and if the South lost they lost. They needed a Confederate victory, regardless of their reservations.
Furthermore, even if they did not desire a Confederate victory, and instead hoped for a compromise peace, and that probably was the preference as it would undermine their political opponents in both north and south, then being very loud about it was unwise.
Much as the final Iraq War vote might not be representative of actual reservations and sentiments among the American political elite in 2002, the later secession conventions after the secession of South Carolina, and especially after the direction of travel in other parts of the Deep South was clear, greatly understate pre-November 1860 doubts. Except, of course, in areas where Unionist sentiment was so overwelmingly strong that it was not just safe, but advantageous to express those views.
But 45% Unionist sentiment tended to collapse to sub 10% very rapidly elsewhere as tends to happen in similar situations.
