My son sounds a lot like yours. He was diagnosed as PDD-NOS at 2.5 years old. He is now almost 5. He has made big strides in many areas, but still struggles with the social stuff. He too, loves adults. I think he would be content if he was never around another child period. He is an only child too, so that makes it a little tougher.
Does your son have any "special Interests"? I mean does he fixate on any one topic more than others? My son is completely into car makes and models. He started into this around 2.5 yrs old, and it has never stopped!
Anyway, welcome to Wrong Planet. I think you will find a lot of support here. I am glad to hear that you are accepting it more now. I think all of us go through a different road, but at some point you have to reach the acceptance stage and just keep loving your child no matter what!