While moderately short, I've never had an issue with other people being taller than me.
I do, however, dislike crowds.
Mainly for the following reasons:
1) I usually have somewhere to go, an agenda. Crowds contain "sunday drivers" - people who just wander unpredictably. Either blocking my path (ignorants!) or moving into my path, forcing me to deviate, bend, dodge, retreat, redirect....
2) Noise. The droning and mixing din of peoples' conversations, the humming, the footsteps... the noise
3) Smells. Possibly even worse than the noise, I have some days where my sense of smell just can't be tamed and I get really jumpy and high strung after encountering so many smells; perfumes, grass clippings, musk, deodorant, cigarette, etc. It tangles me up and makes the crowd my enemy.
There are other reasons in specific sittuations (ie mall crowds vs parade crowds) but you get the idea?