I've noticed that there's an uncanny valley kind of thing that goes on when it comes to group prejudice against the "outcast". If you're so obviously different that you will never be "in with the group" people will tend to leave you alone, accept your differences, and treat you with a distant semblance of respect - especially if you're good at your job. However, if you're close to being like them, i.e. actively trying to fit in but not quite making the bar, suddenly you are fair game to them and they will tease you mercilessly. It sucks but I don't know what else to say other than try not to bother with such people or try to be friendly with them other than in a professional sense. Find the other loners to strike up casual conversation with and just ignore the as*hole crowd.