The Democratic base didn't turn out very strongly. There was a general perception that the race was a foregone conclusion, and no sense of urgency. This might have cost the Dems winning back the House, as well as the opportunity to gain Senate seats and possibly take back that chamber, as well.
Also, allegations of improper campaign contributions to Clinton from China were disclosed in the final weeks of the campaign. Perot in particular hammered at this issue relentlessly. This helped reduce Clinton's margin of victory, and deny him a popular vote majority.
I think youre overstating it. The GOP was never in danger of losing 5+ Senate seats in 1996. Ultra conservative Sen Smith of NH won his race as did Clinton carry NH.
I do think the polls may have been overstating Clinton's support.