Sex matters to men as the single place where they are "allowed" to honor the fundamental basic human needs for connection and belonging. Women like sex just as much as men do, they just have other socially acceptable methods for human connection and interaction. "Feelings" have been forced out of patriarchal masculinity to male detriment because everyone has them and no cognition takes place without emotion. This leaves men emotional cripples who only have sex to turn to for basic humanity. Don't act like this is anything to do with human nature - it's pure socialization in patriarchal cultures.
"Emotions are like the rudder that steers the ship because they push cognition into various modes. This means that you’re more likely to think of things in particular ways, depending on the emotion you’re having. All thinking is inherently emotional and cognitive, all at once. It’s literally neurobiologically impossible to remember or think deeply about anything that you haven’t felt emotion about. It just doesn’t happen because it would be a waste of the brain’s resources to think a lot about stuff that doesn’t matter."
Emotions Are the Rudder That Steers Thinking