Oh yes, I have had that happen more times than I can count. It's a problem I've come to realize isn't going to go away.
The ironic thing, is that as I have become more and more successful in my job, my earlier problems are more or less forgotten, and now when I cautiously mention to certain co-workers (mainly those who have autistic children), that I have good reason to believe I am on the spectrum, they react with disbelief. I don't fit the Rainman stereotype therefore I cannot be ASD. I am working and supporting myself, therefore I don't have ASD. I no longer act like their children, therefore I cannot be ASD.
Meanwhile, at home, among my neighbors, it is an entirely different business. I wish I could give out phone numbers to people who insist that I am normal just like them, because I know quite a few people who would be more than happy to enlighten them about that weird neighbor down the street. What is it that they are seeing that the people I work with are not seeing, and vice versa.
All I know is that as soon as I start believing that I am normal just like everyone else, the universe or god or someone is very quick to let me know that is not the case. This is not of my choosing. On one of the bluest days of my life, after a doctor had told me about a lump, "Yup, could be cancer," I got an anonymous phone call. Now if this was one of those heartwarming inspirational e-mails that we all get, it would have been an old friend calling to say that God had prompted her to call me etc.
Instead, the voice sang out, "ret*d, ret*d, you're a ret*d!" and hung up. This was no random call. This was from someone who knew me well enough to know that that particular taunt would hit home. I never did find out who it was, though I have suspicions. And if there is a God and He is in charge of this universe, then He has a lot to answer for in allowing that particular call to come through at that time.