I get what you are saying and largely agree. We don't have to be outraged by every stupid thing that people do, and it's great to not give other people that much power over you. In the circumstance you described, you probably did the right thing. And, culture-wide, if we never get upset about things that are upsetting, then nothing ever changes. Racism, sexism, homophobia, etc., all continue unabated because we can't be bothered to let them bother us.

I'm not saying we should go ballistic over every little thing, but there are also ways to challenge these things, including outdated notions of what women are capable of, without going ballistic. I stand up to the men who try to bully me for being a woman with a voice on the internet, and now I experience a lot less bullying attempts than I used to. I don't see that as giving them power over me; quite the opposite.