If you decide to go, avail yourself of all the services available. There may be a disabilities center that can provide counseling, training (e.g. in self-advocacy), and other services to improve your outcome. There are workarounds to things like executive function problems - but only if you use them.
Personally, I think it is important throughout our lifespan to grow as a person. You have to take some risks if you are ever to grow. Some of those risks won't pan out, and you need to be able to handle a bit of failure without letting it completely overcome you.
_________________
A finger in every pie.