As I noted to Avery, patriarchy is a social system that women participate in as well. It's not a synonym for men.
However, patriarchy (and male led cultures) are less than 10k years old. For the vast majority of human history, people lived in matrilineal egalitarian enclaves. It didn't matter who fathered a child, because the clan or village took care of everyone in it. The first archeological evidence of war is from only about 13k years ago (about 5% of history). This idea that human history is pretty much just like it is today, but without modern conveniences is a fable.