I don't want to minimize your concerns about injuries, but as Izaak said, you will receive instruction about avoiding injuries and playing safely. Of course, you're not going to be able to entirely avoid getting banged up, at least a little, since that's part of contact sports.
I say give it a try and see how you like it; let your experience, rather than your preconceived notions, guide you. If you don't like it, well, you're not talking about a lifetime commitment here. And how will you know, unless you try it?
I have to admit that I have always gravitated toward contact sports (football, hockey, boxing) because those are the sports I have tended to be good at. Some of that is no doubt based on my build (compact with low center of gravity). Part of it might be an Aspie thing, too; I really don't know. There, I've disclosed my bias which is based on personal preference.
While I haven't played rugby, I know a few guys who have and they've had a blast with it. Also, I have played tackle (American) football without padding quite a few times, so I imagine that's pretty similar. Lots of fun!
Again, I say go for it. Be safe and sensible, but give it a try. I hope you'll let us know how it goes, and good luck with the tryouts and whatever you decide to do with the game!
_________________
All the best to you,
Steve
--
"I can make it, I know I can.
You broke the boy in me, but you won't break the man."
--John Parr, "Man in Motion"