I think very hot weather is worse. In very cold weather, it's easy to get warm with shelter and enough clothes. It's not as easy to get cool in very hot weather, even if you have air conditioning.
That's definitely not my experience. I've been freezing inside buildings (and even f**king buses) even when it was over 90F outside.
I find that most building are far too cold when it's hot outside. It's like someone decided that indoor areas need to be at 60F at all times, regardless whether it's 100F outside or 20F outside.
I like the building cool (55-60 degrees) when it is hot outside (to instantly stop sweating if I am) and nice and toasty (70 degrees?) if it is cold outside. I guess that's an expensive taste. But, constant cold is simply depressing, and heat is not. This week is going to have three straight days in the 40s here, the coldest stretch in a long time. It looks like the 70s should at least return by Sunday.