If it helps, I often feel similarly. I don't think there's really any reason to feel bad about being a late starter compared to the average person, but speaking from my own experience, not feeling sad or bad about it is often easier said than done.
I try to remember that life overall is not a competition--even though it can often feel like one, in terms of getting jobs and such. If all life were just a competition, it would be so much more boring than a lot of it already is, heh.