To a socially acclimated world, social skills is "sense," whereas the "sense" you speak of (the ability to utilize rational thought, I assume) is a useful but ultimately optional accessory skill. More people have innate social abilities than innate logical processing abilities, so the former is seen as more "common sense" than the latter.
I hope that you find the right person or people to make you feel like less of a "freak." If a stranger's opinion matters any, I think you're pretty awesome. 
_________________
I am not a textbook case of any particular disorder; I am an abstract, poetic portrayal of neurovariance with which much artistic license was taken.