But only in patriarchal cultures - where men are supposed to control "their" woman and anything indicating that isn't the case is seen as emasculating. For a lot of people around the world, this isn't that big a deal and I totally agree that consensual activity is nobody else's business.

Polyandry was normal in pre-contact Polynesia, particularly for high caste women and still takes place in the Indian Himalayas and in parts of Tibet. In Lowland South America, and in Africa partible paternity, where two or more men mate with a woman for the purposes of producing a child, is common in many cultures. Spreading fatherly feelings throughout the group helps to maintain solidarity and cohesion as well as promotes the wellbeing of a greater number of children. Reproductive fitness (the chance that offspring will, in turn, produce their own offspring) is enhanced by cooperative alloparenting of this kind where several adults take an active interest in the lives of children.

In most of rural Africa children wander in and out of the homes of unrelated adults in their villages, where they are treated as family. This stems from a more community-oriented lifestyle in general, but also from a lack of concern for parentage because sexual exclusivity is not prioritized. In fact, in some cultures, it is considered downright stingy.

When seventeenth-century Jesuit missionary Paul Le Jeune lectured a Montagnais Indian man about the dangers of the rampant infidelity he’d witnessed, Le Jeune received a lesson on proper parenthood in response. The missionary recalled, “I told him that it was not honorable for a woman to love any one else except her husband, and that this evil being among them, he himself was not sure that his son, who was there present, was his son. He replied, ‘Thou hast no sense. You French people love only your own children; but we all love all the children of our tribe.’”

Ryan, Christopher. Sex at Dawn (pp. 122–123). Harper Perennial. Kindle Edition.