I got my license at 17.....It's been 33 years of terror since. Well, let me qualify that by saying that Interstate driving isn't a problem. It's when I get into dense city driving that the wheels begin to fall off for me.
I think it's a sensory overload issue for me. Parked cars, pedestrians, lane changes, construction, detours, one-way streets, taxi cabs, buses, and too many signs that don't give me enough information when I need it to process and make a decision, etc.
And there's always some impatient ass ready to take me on for the slightest perceived transgression. I find it all too stressful indeed. 