My standard response to this is the simple, factual statement that "I do not listen with my eyes."
Also, I have some issues with processing visual and auditory input at the same time, so trying to read a persons facial expressions while also absorbing what they are saying means I will do both things half-assedly. If they point out that not looking them in the eyes feels disrespectful to them, I will respond that their insistance on having their feelings catered to in this manner is actually disrespectful and unaccomodating to me.
_________________
I'm bored out of my skull, let's play a different game. Let's pay a visit down below and cast the world in flame.