As an adult with Asperger's, I must say it was very rare that anyone would verbally/directly label me as a psychopath, except the bullies back in my early teens or so. But I got the sense, at various points in my life, that people thought of me as a psychopath b/c of my "emotional dyslexia" as we've come to call AS in more layperson terms. Maybe it was because I couldn't find a "middle ground" in terms of my approach in interacting with others - which I've been able to calibrate since - but I was either too overly friendly and familiar (a classic psychopath warning that in my youth naivete I missed, btw!) OR I was too aloof, mumbled, avoided eye contact etc. Classic AS black-and-white thinking you might say. That was the problem, it was more the initial impression that I think people got of me but once they got to know me, if they were enlightened enough, it became apparent to them that this was not the case (in fact, they clearly and correctly concluded that I was far more likely to be on the receiving end of psychopathy!! !)
I remember reading on one online source several years ago that one ill-informed poster described those with Asperger's as "psychopaths who are not very good at it." Also when I was at a lunch event with work colleagues a while back, one woman remarked how she did a BA in Philosophy and said that she'd never want to teach it, because everyone she met who had a PhD in it "had almost no ability to relate to other people, they were almost psychopathic." A fallacious argument, since psychopaths have higher than average ToM (thus translating to higher street-smarts) but virtually no empathy. Whereas people with AS have the reverse (indeed, some have remarked that they have more empathy than others because of a higher moral standard - it's just expressing it that comes across as weird to others).