Like kirayng, I believe it's a sensory difference. NT's seem to be able to screen out what their brains don't need to cope with, but we are hit with everything at once and it's like a kind of shell shock. My brain goes into overload, I become overwhelmed, anxious and panicked.
For example if I have to go to a new place I'm noticing everything in the room at once - people, hair, shirt, space, vibration, light, hum, whirl, white, shiny, reflection, poster, ink, pattern, window, door, green, carpet, smell, paper, rustle, click, talk, words, cough, movement etc
This is what makes it painful. This is why I try to avoid new situations.
_________________
It's like I'm sleepwalking