How masking itself is not the privilege, but the option and choice of when and when not to.
Here's the logic:
Like; you got legs, one that works.
One that can move properly. One that can walk, run, jump, dance, etc.
You don't need any mobility aides, even.
Oh sure, you have that choice.
But do you want to use your legs 24/7 or at least the entirety of your waking life?
Never sit down? Never lie down?

Only run, only walk, maybe only standing? No choice to rest and use a chair?
Do you want to dance for 8 hours straight? If you can, should you??
Only difference is that most people can't just do that, thus it's more acceptable to rest.
But being forced to use your legs past exhaustion? Wouldn't that be unethical to teach a kid to perform for that long or tragically negligent if that's how someone had to survive and make a living?
What is privilege of an ability in being forced into a burden because of said ability?
If you're not allowed to unmask for most of your waking life, that said ability to mask is so dependent on everything you had to do in life, even if you're very torn down or even forget who you are anymore and a path into a destructive route; is that a privilege anymore?
Masking is a survival skill. Without it many of us would be dead already.