Add to that the fact that racists often have things like "this one's different" at times, and xenophobia isn't as simple as "voted for a black guy = can't be racist or ever possibly get swept up in xenophobia ever".
This is beside the point. If you're looking for voters who can't ever be racist or get swept up in xenophobia you're looking at a cool ... three percent of the electorate? Ultimately, you have to be amoral about this sort of thing - as long as they vote for you, it doesn't really matter what they think.
Plenty of undoubtedly racist people voted for LBJ, Carter, Bill Clinton and Obama. They voted that way because whatever their bigotry or bias, ultimately they saw the Democrats as offering something to them, even if they may have grumbled about civil rights or whatever. And ultimately, it was good for civil rights because it allowed Democrats in their post-CRA mode to enter power and maintain the gains of the civil rights movement.