The quote above is from comedian John Mulaney. You can see how popular it is by the upvotes here. I don't know if this was ever meant to be used as a logical argument rather than just for comedy, but either way, I often see people cite it to prove a point. Sometimes this point is correct, but let's look at whether or not it's a cardinal rule.
For example, I've seen this line of reasoning used to prove fat shaming a woman is worse than height shaming a man. It's politically incorrect to merely ask a female her weight, let alone insult her about it. On the other hand, you can talk about a male's height however you want. Therefore, this clearly means fat shaming is objectively worse, right?
Nope. All this proves is that one form of prejudice is currently
considered worse than the other. It doesn't prove why it's
logically worse.
Mulaney’s quote confuses the way things are, with the way things
should be, also known as the is–ought problem. By his own logic, racial slurs weren’t as bad
back when people were allowed to say them, but hopefully we all know that’s
not true.
While racism is considered worse than discrimination towards those with dwarfism, I've seen a lot of people take that as a go ahead to downplay, ignore, or justify the latter. This is called the fallacy of relative privation. Make no mistake, whether you think saying "midget" is as bad as a racial slur or not, prejudice is prejudice.
Now if only people could be as sympathetic about this as the late Roger Ebert. Someone with dwarfism once wrote to him about using the word "midget." After a polite exchange, Ebert immediately agreed to stop using the slur. He also didn't seem to care whether "midget" will ever be as taboo as "nigger," he simply said:
Now wasn't that easy?