I started a new job last year working for a call center of a fashion retailer. When I was going through the interview, hiring and training process, I often had the same feelings as you did. Part of my issue was I had done phone work as a telemarketer in the past and absolutely sucked at it. I knew this job was more customer service than sales, but that didn't help matters.
A little anxeity is normal when you start a new job. Thankfully, I work for an employer that is mostly tolerant of AS, and took the time with me when I had questions and concerns about my work. Do not be afraid to ask management and co-workers for assistance if you need it. Most are eager to help.
Give it a few months, I bet most of your anxiety about the job will be gone.