Defeating Pence isn't going to be easy. Indiana is a very republican state and there is plenty of time for his religious freedom-related moves to fade from voters minds.
This, sadly, is true.
In 2014, State Senator Mike Delph was condemned on all sides for his "twitter rant" following the General Assembly's refusal to ban gay marriage and civil unions in the state Constitution. During said rant, Delph accused lawmakers and several churches of creating a "godless culture". The Republican leadership in the state senate stripped him of his leadership positions and many Republicans - including one of Mitch Daniels' former advisors - endorsed his Democratic opponent (who ran an excellent campaign, I might add). Delph won reelection 55%-45%.