I think it's mostly sociological/patriarchal. A man is "supposed" to be the big, strong, leader of the family, and as a tall woman I can tell you that a lot of guys are intimidated by a tall woman. When I first met James his friend asked him, "Aren't you intimidated by her?" His answer was "definitely not" although he is 2 inches taller than I am.

We typically elect the taller candidate so there is something there that goes beyond the dating/marriage context to ascribing power to height. Women buy into this too, and many only want a guy who is taller than they are. I wasn't willing to cut my dating pool down that much, so I did date guys shorter than I was, but ultimately married one was taller.