Freedom of Speech means freedom from having your speech curtailed by the government. It doesn't mean you have the right to say whatever you want when ever you want to without any consequences. It irks me to no end that so many people don't seem to understand this.

On the other hand, in many states harassment is actually illegal. And you're right that whether or not it is illegal, intimidating someone (a stranger no less) is an unethical thing to do.