In the past year, I've experienced the hottest and coldest temperatures of my life.  I was in southern Arizona in July, and it was around 110°F (if you look at a weather map of the whole country, find the little red dot indicating the highest temperature; that's where I was).  I was recently in Vermont where it was -7°F.  And honestly, neither temperatures were that big of a deal.  When it's hot, you sweat.  When it's really hot, you sweat more.  That's about it.  And when it's cold, you wear layers of clothing.  When it's really cold, you wear more layers.  I think it's common to be a little fearful of extreme temperatures, and for good reason:  You can pass out or get frostbite or die.  But it's more likely that you'll just get used to it and/or go back inside.