Iain King seems to have a good outlook here:
"...there can be no viable formulation of the Golden Rule, unless it is heavily qualified by other maxims."
https://en.wikipedia.org/wiki/Golden_Rule#Cannot_be_a_sole_guide_to_actionThe point is that if we assume that others would want the same thing as we want we could be doing something which we consider good, to others who don't think it good. (for example, would you hug another person without first asking that person if they are comfortable with such an action? Sometimes you may know intuitively that it would be ok, but sometimes people do things without much thought and offend another person without an intention to do so)