If your point is that exclusionary groups are going to foster a less-than-ideal mindset in regards to "out" groups, I guess that's not entirely deniable. It's also going to foster a sense of in-group loyalty that I view as, in many ways, constructive--granting people a smaller, more palpable community than their larger university that they can work to actively build and support.
I think you're overlooking the fact that power dynamics do exist. If you have an in-group that not only seeks to maintain a certain level of privilege above the out-groups, and does so in practice, then, yes, that is a problem.
You say exclusion comes not from race or socioeconomic class, but from that standard trope about being "a good fit." That's the same argument that any overwhelmingly white, male organization will use to justify enforcing homogeneity.
My freshman year of college, I lived in a dorm that fraternities tended to recruit very heavily from, and every semester the various chapters would post rush events on communal bulletin boards and sometimes slip fliers under your door or in your mailbox. The events in question tended to be things like skeet shooting, cigar smokers and going to the driving range. In other words, things that upscale white men tend to enjoy, and things that people who do not come from such a background are going to be unfamiliar with, uncomfortable with and probably avoid. Being from Yankee-and-immigrant stock, I've never felt comfortable around firearms; I also eschew golf and abhor anything involving tobacco; I'm also rather contrarian in general, so that was an avenue that I did not pursue in college.
This might not be a huge problem in and of itself, but when you create a system where membership in said organization can give you substantial advantage in things like getting a job after graduation, you've created a hurdle that can only be cleared by passing through a filter whose criteria is conveniently correlated with being...an affluent white male.
