Can you see what's going on? Here's a good explanation I found:
"Height presents a major problem with what most people consciously and subconsciously believe in: a just-world fallacy. You can't control your height, so they have to work extra mental gymnastics to justify why the problems you have due to your height are not in fact due to your height, but your attitude. That way they can believe it is your fault you suffer, the world is just, and their cognitive dissonance goes away."
"The fact is there is not someone for everyone, and the victims of this did not necessarily perpetrate this condition on themselves. However, acknowledging this would require others to admit that it is not a just world, that what they have was not necessarily earned or deserved, so they'll fight tooth and nail to deny this truth to themselves and anyone else." - Source
More on the just-world hypothesis: "the assumption that a person's actions are inherently inclined to bring morally fair and fitting consequences to that person, to the end of all noble actions being eventually rewarded and all evil actions eventually punished. In other words, the just-world hypothesis is the tendency to attribute consequences to—or expect consequences as the result of—a universal force that restores moral balance."