I don't know why Ryan wouldn't be popular with the base ... the fact that some now call him of all people a "RINO" is truly telling of the Breitbart crowd.
I think it started with him surrendering on the debt ceiling issue and passing omnibus (he was before that adored by hardcore fiscal conservatives). His attacks on Trump even during the general election cycle, support for mass immigration, and (real or perceived) support for establishment republican incumbents over tea party challengers all contributed as well. Trump also instilled in the base a deep suspicion of GOP megadonors like the Koch brothers and (not without reason) Ryan is seen as their puppet.