It would take an act of Congress to expand the House, and more representatives would mean less power for each individual representative. Of course they're perfectly happy to keep the number fixed. That's why there was an amendment proposed in the Bill of Rights that would have limited the population per district. If only the Founders had thought of the cube root rule...
We should all be glad that Madison's harebrained idea didn't pass. One representative for every 50,000 Americans would require a US House the size of a small town.
It just says at least 50k per district. It would allow any number from 200-6600 districts.