If I do look at people's faces I either know them extremely well or I am forcing myself to.
When I force the issue on myself, it makes me think of other things (like their facial features) and I won't be able to listen to what they're saying.
I believe that my inability to look at people gives me face-blindness which means I don't remember them and this, then, creates other issues.
I find it very confronting when people get in too close to me, when speaking, and I always take a step back or turn myself sideways so that they are literally speaking into my ear.
I also have a keen sense of smell and will usually concentrate on people's smells which is very distracting in conversation. Especially so if their odour repulses me and all I'm thinking is: "I have got to get away!"
At least, sometimes, I can remember people by how they smell. 
_________________
Nothing much shocks me...so please stop trying...yawn...