I've seen studies on this and I wish I could remember where they are, but the fear of "if we're winning by too much, people will either stay home or vote third party" is completely debunked. In fact, it's the opposite: people generally feel more compelled to turnout and vote for the winner.
If anything, GOP turnout is going to be negatively impacted if we go into the final week or two with polls showing Clinton winning by high-single-digits or double-digits.
Yeah, this actually makes the rout worse because they want to pull for the winning team. It's called the bandwagon effect