Hitler stamped out the SA, and a bunch of other troublemakers on the Night of the Long Knives, because Ernst Röhm was the only credible threat to his domination. Sure, the SA often sided with workers during strikes, but Hitler wanted to be rid of them because they were getting too big for their breeches, becoming larger in number than the Reichswehr and threatening to win the sympathies of von Hindenburg.
There were lingering radical element like Goebbels but Hitler abandoned most of the more overtly socialist elements of National Socialism in favor of Feder's general dynamic of 'productive' (state) capitalism. Again, there's a reason Otto Strasser and a lot of the old guard fled. Besides which focusing on economics is somewhat irrelevant. Fascism in general is very heavily anti-materialist in character in the literal sense of the term, hence it is somewhat of a mistake to focus on economics and ignore the other recurring philosophical elements of extreme anti-egalitarianism, skepticism/hatred of rationalism, anti-universalism, anti-humanism, fixation on the mythic past, etc. especially in the case of Nazi Germany.
That's not really relevant to what we're talking about though. Nobody disputes that they were both totalitarian regimes.
Except those arguments are very common. It's a favorite tactic of american libertarians and many self-IDing conservatives (of the talk radio variety - see Beck, Savage, etc.) to claim Nazi Germany was leftist in character. Virtually all the big time libertarians have made arguments to that effect, just look at Hayek or Von Mises or a good chunk of Lew Rockwell. It should be noted though that people aren't necessarily disputing that
socialists can be fascist, just that Nazism was left wing.