Hey Joe, if I were elected POTUS, would I be equally "immune" from criticism because to do so would be homophobic? I haven't noticed folks being shy about criticizing Obama by the way (heck even his own party members dump on him from time to time). I guess they don't mind being called racists when they do so or something.
The idea usually is that criticism of Obama is only racist when it comes from the right-wing of the Republican party. Tea Partiers quite understandably get tired of the accusation, and so may express their frustration in unhelpful ways.
The level of vitriol directed at this president is unprecedented. Conservatives didn't like Jimmy Carter or Bill Clinton either, but nothing approached this level of hatred. If you think race has nothing to do with it, you're naive.