Men do absolutely suffer under this patriarchal dominance-based hierarchy (what he refers to as toxic masculinity) that we live in and it's something that I write about frequently, but I think you are fooling yourself if you don't think that women suffer more (not that it's a contest). Not acknowledging all the ways that this is demonstrably and objectively true seems like a way to not have to face it. A mere 60 years ago women were second-class citizens by both law and custom and although those laws are gone, the beliefs they enshrined are still very much alive - manifesting in all sorts of ways that are too numerous to mention here.
All men are not responsible for misogyny as individuals, but the pervasive culture of masculinity in this county is responsible for it (as well as things like racism, because it's a dominance hierarchy system). You as a man not being able to see yourself as a part of a social system is a part of the way that it continues in such a destructive manner, hurting women (and others) as well as men. If you want things to get better for us all, it's going to take dropping the defensiveness and looking at the actual dynamic as a whole.
"Patriarchy is a dynamic based in social stratification that includes not only gender, but race, sexuality, able-bodiedness, education, wealth, and other elements of class stratification. Things like police brutality are a manifestation of the dominance hierarchy aspects. So too is sexual harassment. It’s a way to remind women of their place, as well as a way to exert dominance. What it’s not is a binary where men rule and win in everything and women are always subject and lose. It’s a lot more complex than that."