Men in swinger's clubs, and other sex-positive spaces control themselves all the time when faced with nudity and sexual behavior. They don't feel entitled to it, or go mad with frustration, so other men can do this too. Many just don't want to. They prefer the old narrative that says that women are in the world for their pleasure and enjoyment.