In the past I've had a panicked, fearful outlook like you describe, but I'm trying hard to turn it around!
I'm at a point now where I feel much more confident in my ability to survive in this world. I've been trying to research everything I need to know about future housing options, health care, financial strategies, etc., so I have a good basic plan for how I'm going to live for the rest of my life.
Socially, I am grateful to prefer solitude, so my happiness does not depend on having close friends or family members. (I know not everyone is built that way, and I feel lucky to be strange in this regard!)
My desires are simple - just a roof over my head (a rented room would be fine), budget groceries, clothes from Goodwill, use public transportation, and if I can always have a laptop computer or tablet or some such thing, that will keep me entertained.
I am a minimalist by nature, and having a simple, clear concept of what I truly need in life has helped me to be less afraid, given the reality that I have never been functional enough to work full-time, and need to survive on limited income.
I hope that you can work through your anxiety as well, and get to a place of feeling less overwhelmed!