The normal human frame of vision is horizontal. I presume that's why film (and later digital sensors) on most cameras are oriented that way.
I understand that in addition to frame of vision, the eyes are much better designed to scan (speed-wise) side-to-side. And while our brains know this and normally compensate, there are lots of optical illusions that can eaisly fool us into "seeing" (estimating visually) vertical vs horizontal distances incorrectly.
Personally I often have the vague impression that square format images are just a bit taller than wide, even when I know it's not the case. Possible though it's the camera format of most 6X6 systems, but I suspect it's because of this type of bias.