I've also always been bad at sports - as a child I was diagnosed with a "delay of motor development". The only kinds of sports I like somewhat are swimming and badminton (I also like hiking, but I don't really consider that sports). I've never had the chance to do sports with a group of people whose skills where just as lacking as mine. But since the beginning of this grade, I'm in a Physical Education course with different students and a new teacher - and while I still don't like it, I can stand it now.
I think that's partially because my skills have improved and partially because I'm now playing sports with people who are quite considerate - the teacher is aware that some people are just bad at sports and it's not laziness and there are two girls who help me by gently correcting my mistakes and make me feel less self-conscious.
Now, during a ball game, there are moments where I actually know what to do and actually think it's fun. I think it's similar to your experience, Chickenbird - "No wonder so many people love sport!" (I don't write "NTs" because I don't think being bad at sports and therefore disliking it is a problem only people on the autism spectrum can have. I, for example, might not be fully NT (I don't know), but I'm quite sure that I could not be diagnosed with Asperger's or anything related.)