Libertarianism strongly emphasizes freedom of choice and individual decisions in political, economic and social settings. This can be closely paired with the concept of self-reliance.
Self-reliance was a core principle of Lincoln's and so it also became part of the thinking of the early GOP. During the Great Depression, the GOP again turned towards self-reliance as one of its counterpoints to the New Deal. Reagan built much of his campaign on themes of self-reliance. Given how self-reliance is so deep-rooted in the Pub philosophy and so easily associated with libertarianism, it isn't surprising that libertarians tends to be viewed more as right than left.
Bingo. That's why it's so dumb to look at things like "who supported states' rights at the time?" or "who supported infrastructure spending?" It's the motive (and not the method) that's important to deciphering ideology, IMO. The GOP of Lincoln's day supported federal spending on necessities like railroads to allow entrepreneurs and skilled, talented Americans to thrive in a changing world; the GOP of Romney's day opposes increased federal spending on things like welfare because it sees it as folks taking advantage of the system and not choosing to lift themselves up. Whatever one may think of that last viewpoint, the two are perfectly compatible, and both fit with the broadly "conservative" economic view that has prevailed in one party or another since the Federalists; Libertarians largely share this philosophy.