And all of those categories benefit you at the expense of other more marginalized people - even if that isn't something you want to be happening. You can be trying your best to foster equality and still be cognizant of the ways that your privilege benefits you and disadvantages others. You don't need to rend your clothing and tear your hair in atonement - most of that stuff is just the luck of the draw. But what you might want to do is put energy into educating yourself, and trying to be a good ally. Exchange "I didn't do anything" for "what can I do to improve things" and then just do your best with that.

Edit: Mark, this was more directed at the other comment than at you, just FYI.