Also, you had more "ghettoization" outside of the South, ironically. There was little or no de jure segregation outside the South, but a huge about of de facto segregation.
It was not uncommon, in many southern locations, for a small shotgun house owned or rented by a black family to be within sight of, or even next to, a mansion of a white person.
Hence, a friend of mine, who is black, had a 5th great grandfather who was literally the next door neighbor of John Tyler. (Tyler, while a Senator, helped the ancestor get a pension for service in the American Revolution.)
Yeah its interesting. When the black caucus was first founded, none of them were from a state of the old confederacy. They were from:
Oakland (Dellums)
South Central LA (Hawkins)
St Louis (Clay)
South Chicago (Metcalf)
Detroit (Diggs and Conyers)
Cleveland (Stokes)
Baltimore (Mitchell)
Philly (Nix)
Big Apple (Rangel and Chisholm)