It is stressful.
Hopefully, you and the caregiver can come to an understanding where you list tasks for said person at the beginning of the day, and you allow the caregiver to do the caregiver thing and you do your thing with the caregiver giving you ample warning of when your "space" is needed. It often will take time to get the caregiver to understand you want more privacy and less interaction than most people. Some will never understand. Hopefully, you do have the ability to say to the insurance company or the agency the insurance company has contracted out to that this person is not working out please send another one. I have never run into a situation where I did not have that option.
_________________
Professionally Identified and joined WP August 26, 2013
DSM 5: Autism Spectrum Disorder, DSM IV: Aspergers Moderate Severity.
“My autism is not a superpower. It also isn’t some kind of god-forsaken, endless fountain of suffering inflicted on my family. It’s just part of who I am as a person”. - Sara Luterman