FWIW he's said publicly that he'll recruit/help organise a primary, but would have his name on the ballot as an absolute last resort.
That's correct. He's said repeatedly that he'd rather help someone else challenge Trump than run himself. Maybe he's just playing coy, but my hunch is no, he'd only run if, say, Kasich, Flake, Corker, Sasse, etc. all decide not to run, and the only folks willing to challenge Trump are people like Brad Thor. In that circumstance, I think Kristol might run.
EDIT: I guess it's also possible that Kristol would run if someone like Amash was the only other challenger, since I'd imagine that Kristol wouldn't be able to stomach Amash's foreign policy.