Haha, yes, I do look at the camera far more often than at the other person. Or if I'm skyping and working at the same time, I'm looking at an app on the screen.
I always think it's more polite, like looking at someone during a face-to-face conversation. Yet, almost everybody I know has told me it looks unnatural, and it makes them feel uncomfortable.
The real question is: why do tech companies advertise their awesome conference video call apps with images of people looking at the camera? Because that is where I learn these things.