I used to do this a little and my brother did it a lot.
I am an aspie and he had ADHD.
Never bothered me, but one warning that my brother never mentioned as a kid...
He says now as an adult that he used to have terrifying episodes where he was lying in bed paralised and couldn't move, but could see. Stuff of nightmares he says. I assume his eyes and brain were waking without signaling his body to remove the normal sleep blocks on movements.
Just a warning to be sure he is actually asleep, as my brother apparently found this very distressing, but never mentioned it. Possibly a bit of theory-of-mind issues of his own.