I have definitely noticed since we opened up our marriage and I no longer feel constrained or guilty about finding other people arousing, I look at them differently. Any crowd is a smorgasbord of fantasy and I like to play a game when James and I are out where we wonder aloud what certain people are like in bed. "See that slightly chubby waiter over there. He may not have the hardest body in the world, but he likes himself and he's got a mischevious twinkle in his eye. I'll be he's a lot of fun in bed." That kind of thing.
I've also seen research that was done by attaching sensors to genitals (and not just self-reporting) which found that most men were aroused by what their sexuality would predict, but women were aroused by anything and everything, including bonobos getting it on.
I completely agree that positioning men and women as opposites rather than as very similar and all human beings just sets us all up for a world of issues that contribute to sex that's not as good as it could be.