You'd also spent your whole life (I presume - correct me if I'm wrong) steeped in a culture that objectifies women's breasts/bodies, so being on a clothing optional beach was not the norm for you. African (and other) tribal cultures where women never wear tops do not experience that turmoil - or it wouldn't be their everyday culture. Anatomically modern humans didn't wear any clothing for the first 90,000 years or so. This is demonstrably NOT a human male nature thing. It is only a wiring thing in that your culture has taught your very plastic brain how to react.

Historians reckon that women only began to cover their breasts in public about 3500 years ago — when men decided they were private, sexual body parts that needed to be tucked away. Prior to that, artwork from the era suggests that women were able to wander around topless without anyone batting an eyelid. Even during the days of Ancient Egypt, historians believe that women could choose whether to cover their breasts or not — often opting for elaborate dresses and designs that would leave one breast exposed.It’s been noted that as women were more and more forced to cover up in Ancient Greece, the more patriarchal the society became until women were fully under the authority of fathers and husbands.

Dr. Conrad Manning speaks to this in his paper Virtues of Nakedness: Physical & Psychological Health.

Sociological studies of ethnic groups in which nudity is common have demonstrated that there is no cause and effect relationship between being naked and immorality. Social nudists also exhibit this lack of association. Still, it is clear that many people perceive an association between nudity and overt sexuality. From the standpoint of scientific psychology it can be demonstrated that this connotation of nudity is no more than a learned sexual fetish. It is not an innate [human] characteristic. (emphasis mine)