I can understand that your family might be trying to avoid another battle and are trying to take his sensory issues into consideration, but I agree with the other comments and think that this is something you all need to start working on.
Maybe just start out with having him wear shorts and a loose tee shirt (with no tag) around the house. You could make him part of the process and have him pick out things he likes that he’d want to wear, so he’d feel less like he was being made to do something he doesn’t want to do.
With my own kid, I’ve found that trying to think of these sorts of things makes all the difference. It’s great that you all are concerned about his comfort. It sounds like he has a lot of support.
_________________
Better to reign in Hell than serve in Heaven. – Satan and TwilightPrincess