Of course it happens and there may even be men who feel real fear in the face of that, but it’s much more likely that women experience fear around men who are behaving angrily towards them.
“In truth, this instinctual terror really isn’t fair to men, the kind and considerate men who really wouldn’t hurt a fly. The culture of patriarchy isn’t fair to anyone. It’s sad for both men and women that we can’t meet them with an unhelmeted face, no matter how trustworthy they seem. Especially when boys are often taught that anger is the only expressible emotion, it must be terribly difficult to find out that even your mild anger causes panic in people you love. It sucks when people think you might hurt them and you wouldn’t — but it sucks way more to live in deeply ingrained, constantly reinforced fear of being hurt.”