Its proximity to the Augusta metro belies the fact that it's largely rural. In counties like this, the Black population has generally been moving to larger cities, while the white population, which controls more of the wealth and resources, stays. In addition to the shrinking minority population, white voters typically turn out at higher rates, skewing the voter population away from the demographics of the county at large. I believe that there's at least one other majority-minority county in the state that voted for Trump (Dooly), which in fact voted for him both times.
Biden did worse than Clinton and Obama throughout much of the rural Black Belt. In addition to Burke County, Trump also flipped Scotland County, NC and Clarendon and Dillon Counties, SC. And a number of Black Belt counties in other Southern states (i.e. Charles City County, VA and Jasper County, MS) swung towards Trump as well. So it seems like Democratic strength is stagnant or is declining across the Black Belt, for the reasons you identify.
