Yep, that's pretty much my experience too. When I was a child I just kind of drifted in between the other kids, ignoring them, pretty much oblivious.
I don't think I started interacting in a relational fashion until around the age of ten or thereabouts. Before that it was mostly requests, demands, responses to others' actions or to others' questions, etc.; functional communication rather than interaction for interaction's sake.
Nowadays I interact readily, but I still don't really form attachments all that easily. It's not that I'm shy; I've actually got no problems with talking to people, even doing public speaking; but I don't really have a people-focused life. It's an odd variation of the "aloof" interaction style that people talk about all the time as being present in autistic kids--I interact; but it just doesn't feel like a significant activity to me.
My main interaction is with the world of information...
Oddly enough, I actually care about people deeply, because people are almost like specialized, fascinating, sentient pieces of information to me, just like I myself am a sentient collection of information. In fact, I first started interacting with other people willingly when I realized they had information I did not have and could not obtain from books. To me, every human being is a little universe, and communication is the only way to transfer information between the two spaces...
I care about people I do not know nearly as much as I care about people I do know, and about people who I have never met nearly as much as I care about people I see daily. I can be distressed by the misfortune of a perfect stranger just as I can be distressed by the misfortune of someone I know and like. I don't know what this says about me. Maybe I'm just not really attached to the rest of the world socially... maybe I just don't see why "Person I Know" is supposed to be fundamentally more important than "Person I Don't Know".