Sol is, as usual, correct here; among great American cities, the only one with a location as strange as Atlanta is Dallas. The cities on the fall line in Georgia are Augusta, Macon, and Columbus.
One disadvantage that the fall line cities that far south have is that the weather is presumably pestilential, which would explain why Piedmont cities like Atlanta and Birmingham would have become relatively more important once the railroads made them possible. That doesn't explain why Atlanta got so much bigger than Birmingham, though. Instinctively I'd expect the opposite.
This is a really good point that I never connected 2 and 2 on prior. Even in my region of the state (NW GA, which is clearly in Appalachia by any definition), a lot of non-mountainous areas are only 700-800 feet above sea level; most of the original Atlanta Piedmont area is at 1000-1200 feet. Macon, Columbus and Augusta are all at 250-400 feet by comparison. Even today, travelers visiting these 3 cities in the summer should be expected to be accosted by inch-sized bugs (along with all of the tinier varieties of pests) when outdoors - something that doesn't really exist anywhere in the northern half of the state.
A lot of people don't realize that much of the original parts of Atlanta are at the same elevation as places like Roanoke VA, Phoenix AZ & Cleveland OH. Just comparing to Macon (60-70 latitudinal miles south of Atlanta), the typical Atlanta temperature is 4-5 degrees cooler with humidity 5-10 points lower year-round; of course, depending on where you live in ATL and how much asphalt and development exists, this might be offset to a large degree.