Who told you that patriarchy is only about gender? It's not - as I've already said, it's a social system based in a dominance hierarchy that is about social stratification of all sorts which is maintained by ruthlessness. Also no, it didn't change strictly with the shift from tribal living to agriculture. Çatalhöyük and the other large proto-agricultural settlements in the same area lasted for thousands of years with an egalitarian system - meaning they were non-patriarchal. Yes, they had gender equality, but they also had no leaders and no class stratification of any sort. That's the true demarcation between patriarchy and pre-patriarchy.
My stance only seems incompreshensible to you because you are mentally locked down into old school thinking about topics that are a lot more nuanced than you have been led to believe. Patriarchy constantly justifies itself. There are 6 non-patriarchal cultures in the world today as well as several others, such as the Danish, who are essentially non-patriarchal and highly invested in more partnership-oriented ways of creating their society.
I've written scores of stories about all of this so I'm not going to continue writing here, but I encourage you to read at least some of them if you want to know the truth and not the propoganda that you've been fed.