FWIW I don't really think "it's more rural than suburban" even if true is a good explanation, demographically it's not like most rural counties at all. It's the 131st highest income county in the country which places it in the top 5th percentile and over 38% of residents have a college degree.
I think the two big factors are: 1-the fact that it's still superficially rural means that it doesn't attract people who are culturally liberal even if educated, (another similar place in NJ by the way is Colts Neck) and 2-check out the age distribution, it skews pretty old.
What I found surprising is it actually lost population last Census, not common in these type of exurban areas. One theory I've heard is that there hasn't been much new development there, and kids who grew up there and moving away and not coming back leaving empty nester parents who thus have a smaller population household, and families with kids aren't moving in to replace them.
Interestingly, I have a friend my age who bought a house with his fiancee in Stanhope for $300,000 and has lived there since the end of last year. Real estate is much cheaper there compared to most of New Jersey, and he has an acre or more of property. I'm not entirely sure what property taxes are like there, though. They're probably outliers though. I'm not sure too many other millennials are moving there.
And in my experience, I would consider it exurban overall. I've only really ever been to the towns of Byram, Sparta, Stanhope, and Vernon but those ones feel fairly close to any other suburb in the state, honestly. It's just that they're more sparsely populated and further from the city. Certainly there is also a lot more space and woodland areas though.
