You are entitled to your views, of course, but my point is that cognitive science says only about 2% of thought is conscious and the rest is unconscious, made up almost entirely of stereotypes, cultural narratives, childhood indoctrination, etc., and that a lot of what's there can even be in contravention of conscious beliefs. So someone can honest to goodness not want or intend to say something offensive and then blurt something like that out because it comes from that place deep inside that has absorbed the racist culture. "Jokes" like that often fit that bill. That doesn't mean they should be branded a racist necessarily, but I think it's still important to identify that this kind of casual comment really is offensive.

And I appreciate that you want to give people the benefit of the doubt.