Can you see what's going on? Here's a good explanation I found:
"Height presents a major problem with what most people consciously and subconsciously believe in: a just-world fallacy. You can't control your height, so they have to work extra mental gymnastics to justify why the problems you have due to your height are not in fact due to your height, but your attitude. That way they can believe it is your fault you suffer, the world is just, and their cognitive dissonance goes away."More on the just-world hypothesis: "the assumption that a person's actions are inherently inclined to bring morally fair and fitting consequences to that person, to the end of all noble actions being eventually rewarded and all evil actions eventually punished. In other words, the just-world hypothesis is the tendency to attribute consequences to—or expect consequences as the result of—a universal force that restores moral balance."
"The fact is there is not someone for everyone, and the victims of this did not necessarily perpetrate this condition on themselves. However, acknowledging this would require others to admit that it is not a just world, that what they have was not necessarily earned or deserved, so they'll fight tooth and nail to deny this truth to themselves and anyone else." - Source
Also related is Surviorship Bias:
Whether it be movie stars, or athletes, or musicians, or CEOs of multibillion-dollar corporations who dropped out of school, popular media often tells the story of the determined individual who pursues their dreams and beats the odds. There is much less focus on the many people that may be similarly skilled and determined but fail to ever find success because of factors beyond their control or other (seemingly) random events. This creates a false public perception that anyone can achieve great things if they have the ability and make the effort. The overwhelming majority of failures are not visible to the public eye, and only those who survive the selective pressures of their competitive environment are seen regularly.People love pointing out short men who are successful socially, romantically, or financially. This ignores the adage of "life isn't fair." People love to believe life is cut out evenly. If there's something for this short man, there must be something for that short man. Wrong. While we should all strive to be at our best, we must also not be unrealistic. Karma isn't real. More importantly, we must be careful with where we assign blame.
As one Dr. Berscheid says: "there is a hazard inherent in denying the impact of physical attractiveness: ''Unattractive children who are unpopular may wrongly attribute their lack of popularity to some flaw in their character or personality,'' she says. Such an error, she believes, could result in lasting and painful scars."