My first memory is being held by my mother. Which isn't a bad memory. She tried to show affection to me as much as she could within the confines of my family (not going there). I only liked her holding me. Which continued as I aged, and transferred to only letting people I trust hold me. I like affection, but I need trust there. I was made a "trust" and "mistrust" person from an early age that continued on as I aged by being that bad luck magnet, where I see the latter until it's proven otherwise, regardless of what my gut says (my gut is alright in determining "good" from "bad", but it's only instinct).
I got adequate affection, but it was only from one person. Well, adequate for someone with autism.
That trust transfers to other contexts all the same. If I trust someone to hold me out in that real world, I'll trust them with things I don't talk about. It takes a long time to build that trust in reality. Hell, I need some form of basic trust just to interact with people socially.
I have no idea how everyone just trusts everyone else so easily, but then, maybe they never needed a reason to question it. I guess if you haven't been stabbed in the back you don't really watch your back all that much.
I have sensory issues too, but they aren't too bad comparatively speaking.