That’s really awful, like he was trying to bait you. I have trouble imagining how must be to have to listen be to have listen to such devaluation. As it happens I know a lot of black people and they all relate experiences like that.
There are a lot of racist tid-bits like that floating around. My mom once told me that black people possess an “inborn sense of rhythm.” There are many more like that but there would be no point in enumerating. When I hear that sort of thing now I’m embarrassed to have a nominal connection to the speaker is all. It doesn’t cut because I’m not on the receiving end.