Then congressman Richmond comes up and mentions the 3/5 thing clearly without understanding that the purpose of it was to stop slaveowners from getting more power.
It is not at all clear that the alternative was for slaves to be counted as whole persons for purposes of apportionment. It is entirely possible that "all other persons" would not have been counted at all. After all, the compromise originated in the Articles of Confederation, where it related not to apportionment but rather the amount of tax each state owed to the central government (which was determined by population, with slaves counted, after the compromise, as three-fifths of a person). So the slave states initially favored slaves not being counted at all.
