Our Gemara on Amud Beis gives examples of hurtful speech that taunts and belittles:

If one is a penitent, another may not say to him: Remember your earlier deeds. If one is the child of converts, another may not say to him: Remember the deeds of your ancestors. If one is a convert and he came to study Torah, one may not say to him: Does the mouth that ate unslaughtered carcasses and repugnant creatures, and creeping animals, comes to study Torah that was stated from the mouth of the Almighty?

The taunt given to the convert is incongruent.  In comparison, the taunt given to the penitent, and even the taunt to sons of converts, involves a true accusation.  The penitent obviously sinned, and the son of converts did indeed have ancestors who committed sinful acts, such as idolatry etc.  However, why should it be shameful to the convert that he ate non-kosher prior to his conversion, as he committed no sins at that time?  The Mei Hashiloach (Likkutim Bava Metzia 58b) has a different take on the accusation. He says the convert is offended because actually he still may commit these sins, and certainly has a propensity to do so.  Mei Shiloach says, like all insults, they only hurt because there is some truth to it.

Not that anyone should have license to hurt another’s feelings.  However, when we become insulted, we should ask ourselves, “It hurts, but is there some truth to this?”  Tiferes Yisrael (Boaz Avos 4:1) says that a person’s enemies are the people that will give you the most honest rebuke (because, due to their animosity, they don’t hold back), and so it is worth listening to them.

One of my favorite and timely quotes from Tucker Carlson is, “They don’t get angry at you for telling lies (even though they claim you are lying). Lies aren’t dangerous as most lies will eventually be discovered.  They get angry at you when you say inconvenient and threatening truths.”