FWIW I don't really think "it's more rural than suburban" even if true is a good explanation, demographically it's not like most rural counties at all. It's the 131st highest income county in the country which places it in the top 5th percentile and over 38% of residents have a college degree.
I think the two big factors are: 1-the fact that it's still superficially rural means that it doesn't attract people who are culturally liberal even if educated, (another similar place in NJ by the way is Colts Neck) and 2-check out the age distribution, it skews pretty old.
What I found surprising is it actually lost population last Census, not common in these type of exurban areas. One theory I've heard is that there hasn't been much new development there, and kids who grew up there and moving away and not coming back leaving empty nester parents who thus have a smaller population household, and families with kids aren't moving in to replace them.
Urban, Rural and Suburban are sometimes very relative. I know I have a skewed idea of what it suburban or urban, since my hometown has a population density of around 5,000 per square mile. It's my idea of a suburban area, but that density to many other Americans would seem quite urban.
Sussex may not be typically rural but it absolutely feels that way to people from the rest of the New York metro area, including myself.