The sort of people basing their vote on email server maintenance weren't going to be voting for Clinton in the first place.
Nobody is basing their vote on the scandal itself as much as what it says about her character, especially as she continuously makes attacks on Trump. I've never heard the expression "two evils" used so often and so seriously in any election in my lifetime. People might think Trump is a joke, but they certainly don' t view Hillary as this shining beacon of clean politics and honor.
I don't know a single person that would have voted for Clinton before that is refusing to vote for her now because of emails. Also, people don't view Clinton as a "shining beacon" because she has faced 3 decades of polarizing personal attacks from the GOP. A sustained campaign like that is going to impact public opinion, especially during election cycle.