Probably an 8 to 12 point loss. Though Romney could have won had he been actually running because he did have accomplishments to run on that year. He had just signed the health care bill into law and could have ran on a platform of ensuring it's implementation, preserving the budget surplus he had created and maybe some education measures or something.
Without him running for reelection, there was a stream of anti-Romney press to help Patrick, with little or no response from Romney in his own defence. His approvals weren't all that terrible untill late in 2006.
http://www.surveyusa.com/client/PollTrack.aspx?g=454ada23-20dc-41f0-9c0c-a8a2a45d653cHe was actually in postive territory as late June 2006, with 56% Approvals. Edit: Misread the color code (I have always seen and used red for disapprove numbers since they are net negatives) and the date, lol fail. He was at parity in July, 48-48. Still very much a winnable situation.