I doubt the students who were 21 or older were voting in their college town anyway. Consider how much more tedious it was to register to vote and change addresses in the era before the Internet and computers. They probably just voted where their parents lived.
Generally, only seniors would be 21 before November, and this would only occur in half the years, so it was not a particularly large population to target. There were often laws that prevented students from registering where they were students. Residence is established by a combination of physical presence and intent. The cases overturning these laws were based on the State not being able to presume intent based on activity.
e.g. An 18 YO freshman might be just as likely to be in the same county two years hence, as a roughneck in North Dakota.
