quote]
Nope, not at all! She's lovely and I've told her all kinds of personal stuff (in Japanese). If I don't like someone's face I will purposely avoid their eyes. But that's normal, right? NTs do that?[/quote]
Well, just because NTs do it doesn't make it normal, but I think you know that.
I don't avoid eye contact with ugly people, but if the person is sleazy I often will. I'll avoid an overly aggressive salesperson or someone who is behaving very inappropriately (not quirky, a pervert, drug addict while they are very high, or something like that).
In Japanese culture it's polite to avert the eyes so you're probably not offending her. It sounds like this is something you need to do to focus on your language aquisition or you've somehow picked up on Japanese norms and are subconsiously mimicking them.