I'll speak for myself here.
Growing up, I knew I was different. I stayed away from most people; was alone most of the time. Loud noises made me jump out of my skin. I couldn't relate to people's fashions, behaviours, attitudes, etc. They saw me as a weirdo but I was in my own little world. While other people were out having sex or whatever I was deep into invention and research. I read anything and everything I could get my hands on and collected things from subway maps to euphemisms for "said" and "walked" to be used in my writing. Tried to learn Latvian. Started reading the Cambridge Unabridged Dictionary from cover to cover but stopped about 3/4th of the way because I discovered electronics and the guitar. While everyone was walking this way, I was walking that way. It's no surprise I only had sex once in my life, when I was 37 - 16 years ago, and it was with a guy. I rocked incessantly when I was a kid; these days I tap my feet often which, I've been told, irritates people. Because it's near impossible for me to make and keep friends, I don't even bother anymore, especially since I'm very prone to saying off-the-wall things that makes people disappear anyway. I'll certainly go to my grave as a homeless, jobless, childless individual.
Later on, lo and behold, I discovered all of these differences had a name - autism, and I didn't know that till I was 49. I suspect a lot of people aren't aware that there is a term for such behavior which, to me, don't seem odd at all. All I knew was I was a misfit - and still am.