I think it is because there is a lot of prejudice against people with disabilities. There's already something different about you that they don't like and therefore they look for the most minute reason to jump down your throat.
Case in point: sometimes my coworkers do not do their job correctly, stand around slacking off, come in late to work, and management doesn't say a word to them because they are the manager's buddies. With me, on the other hand, if I make the slightest mistake, for example: standing around (not to slack off, but to be in the vicinity of a customer), coming back from break even 1 minute late, not having perfect looking (without bothering to look at the actual results) job performance, I get reprimanded, sometimes right on the sales floor. I think it's because they know I'm autistic and have accommodations in place for such, that they sometimes treat me like I should be perfect, while paradoxically treating me like I'm less capable than they are and like I'm beneath them.
My advice: just do the best you can and give 100% to what you're doing. That's all anyone can ask of you. And as for people who expect nothing less than perfection, they have the problem, not you.