I can speak to the North Carolina cases, and hopefully that will prove illustrative.
First of all, Asheville isn't a college town in the way that these other communities are college towns, since UNC-Asheville is a small liberal arts school with a relatively light footprint. Asheville seems based on vibes like it would be a college town, but it isn't really.
In the case of Chapel Hill and Durham, there are several factors at play. There are going to be some people who prefer living in more rural places but who share cultural preferences and interest with urbanites. Towns like Hillsborough and to a less extent Pittsboro are pretty attractive if you want a more small town lifestyle but want to be able to go to see world class musical acts on Duke's campus or at Cat's Cradle in Carrboro. The result is that these surrounding communities have become quite left-leaning; Hillsborough iirc has one of the highest percentages of writers of any town in the country. Even rural areas around Chapel Hill are highly left-leaning; you have lots of organic farms, ecologically optimized farmhouses, etc.
The other factor is cost. Chapel Hill has always been a fairly wealthy community, partly because it's a bit of a suburb in disguise; it actually has a lot of residents who commute to Durham, RTP, Raleigh, etc. and who live there for the high quality of services and extremely good municipal schools. So it's always been a bit pricier than is typical for North Carolina. Durham used to be very cheap but has seen a ton of gentrification in the past 15 years or so. Neither is New York expensive but the area has gotten a lot less affordable. The result is that a lot of people working lower-paying jobs can't afford to live in Chapel Hill anymore or Durham for some.
Consequently, a lot of these people live in farther out exurban or quasi-rural communities; places like Northern Orange County, Mebane, rural Chatham County, or other parts of Alamance. This is amplified by the fact that Orange County has pretty restrictive land use, meaning there aren't as many large-scale housing developments in the northern and western parts of the county, which reduces availability and pushes people to towns in Alamance. These folks are still moderately Democratic leaning, so this helps to extend the "sphere of influence" you talk about.
I think these two patterns obtain in a lot of college towns, at least in the case of large schools with more liberal cultures. The counter-examples you cite are either conservative schools (more on that below) or schools in massive metropolitan areas. (Irvine is actually quite Democratic too, more than is typical of a far-flung Orange County suburb!)
Also ig another question is what makes some colleges so much more liberal than others, even once you account for if a school is religious and region. Texas A&M vs U Austin is a prime example that ik has been discussed before.
Well, as you say region accounts for most of it. Another big element is land-grant schools vs. non-land grant state flagships; the former tends to skew to the right of the latter since it entails having a lot of kids studying ag for their family farms, engineers, etc. as opposed to liberal arts majors. This doesn't usually matter as much in more liberal states (NC State and Iowa State still vote quite D) but in states like Texas that actually matters quite a bit.
Another big factor is historic counter-cultural presences. Austin has long had a rich arts and social activism scene; that's not true traditionally of College Station.
