No you don't. I have a non-verbal autistic son who is 25 who will live with me for the rest of his life. Figuring out who will take care of him after we're gone has been the hardest and most scary thing of my life. I understand more about your life and world than you give me credit for.
And at the same time, all people in a culture are either a part of the problem or a part of the solution. My son is not a part of the problem because he doesn't watch porn, he doesn't laugh at misogynistic jokes, he doesn't watch movies where men treat women badly and otherwise uphold patriarchy. That's probably true of a lot of your friends as well. Great!
North American masculinity is a monolith, and a toxic one at that - it's called the Act Like a Man Box. That doesn't mean every single guy buys into those norms fully or to the same extent, but every man is exposed to them and feels pressure to do so - even you and your friends. You are not separate from your culture. You are deeply and mostly subconsciously influenced by the world around you. You don't get to opt out of responsibility for helping to co-create something better due to disability. You either help or your make it worse. Period!
And for the record, don't misrepresent me - I always say "some men" or "more men" - I always recognize that there are guys out there fighting the good fight. Lumping me in with other people who don't do that is what isn't fair.
Also, Sweden has a high rate of rape because until not that long ago they too were a patriarchal culture. Despite some gender equity, they still have a long way to go. The reporting levels and way that they determine what constitutes rape are largely incidental to the point of this discussion. Honestly, this is one of my areas of expertise.