I get that. But since Nazism is affirmatively genocidal I do think it poses a danger to a civil society. We have so many other ways that our speech is legally curtailed in the US other than just the one that you named. For me, Nazism is essentially incitement to harmful behavior or beliefs that foster harmful behavior. It doesn't get treated that way in the US because you have to prove clear and present danger, but I think that's semantics when someone has genocidal beliefs. JMO
https://medium.com/inside-of-elle-beau/free-speech-is-not-an-unfettered-right-5061d745ea32
Incitement — The Supreme Court held in Brandenburg v. Ohio that “advocacy of the use of force” is unprotected when it is “directed to inciting or producing imminent lawless action” and is “likely to incite or produce such action.” The element of clear and present danger was later added in to clarify this ruling. Incitement to suicide is also not protected speech.