I don't really like telling people I have Autism, because if one says this, one has to somehow prove it. For instance, I was sitting in the library reading, and a man came up to me and started to engage me in talk, asking about the book I was reading and so forth. After a bit he asked me 'what I did'. I said I didn't work because I had Autism. He seemed very interested by this and immediately started quizzing me with a view to (as far as I could tell) creating a trap that would make my claim look spurious.
He: 'What does it mean then - I mean, how does it affect you?'
I: 'It means I'm not very good at social interaction.'
He: 'But you're talking to me now.'
I: It also means that I am hyper-sensitive to sensory stimuli, like noise and light.'
He: 'But there are some children shouting down there' *points to children's play area* 'and you seem fine.'
I can't remember the rest, but it continued for a while in the same sort of vein. It annoys me that people feel the need to challenge me, as if to suggest I'm just pretending. I know it's not as if my explanation of Autism was very good or comprehensive, but still.

I think it's mentioning not working that probably caused that. People get very self-righteous about that, and if you're not in a wheelchair with brain matter leaking out your ears, they're not going to accept that you can't work. So, they're going to treat you like a lying, faking, slacker. Maybe tell such people it's brain damage, and that you don't feel like recounting the details your medical history to a stranger.