Uh, actually I don't believe that anyone has ever said that. Clearly, everyone has problems and faces hard things and all sane people realize that. Some people just object to every single conversation being turned back to that fact - yes, all lives matter... we know that, but now let's spend a few minutes talking about the particular issues of those of us who have been more systemically marginalized and oppressed.
I will agree with you the systems that our society is structured in are much more important than the demographics of those who happen to have been born at the top of the dominance hierarchy (white, cis, males). In other words, men aren't the problem, the patriarchal dominance hierarchy is the problem - but that social system is being vehemently upheld by white cis men, so there's that.... although women and other races uphold it as well, and it harms everyone.
And there are far too many people near the top of the dominance hierarchy who feel they don't need to listen, but instead need to insert their "opinions" around experiences they have never had. In my personal experience, these "opinons" are often completely out of alignment with the lived experiences of marginalized people. Perhaps that isn't something you personally do, but it is pervasive.
The way to decrease suffering is not to demonize "victimization" but rather to let those who have been harmed take the lead as to what they would prefer to be done in order to address their concerns. It's also for us all to take a real look at the social system that underlies all of these issues and confront it.
