Describing Jefferson as "a loud man" is certainly a novel take, considering he was famously introverted and as a member of Congress rarely if ever spoke during debate (unlike Madison). I agree that Madison's misappreciation of partisan politics is a major reason he was less successful than Jefferson or Hamilton as a national leader. (N.B. I don't believe Madison ever served as governor; you're thinking of Monroe.)
*Corrections:
First, when I say loud, I do mean dominating. It’s always shocked me that Jefferson gets so much more fame (and statues) than the man who fashioned our very government. Secondly, you are correct that I misattributed him as Governor. He was a state delegate and a member of the Governor’s Council of State when he wrote about his frustration with the lack of state executive(gubernatorial) powers. My apologies.
I would like to add onto this conversation as at one point Madison was quite close to some leading Federalists, He was one of the coauthors of the Federalist papers with Alexander Hamilton and John Jay.
Federalist Party =/= Federalist Papers.