I think algebra is more about general logic, so I could understand some people finding that easier than doing a lot of calculations.
For me a lot depends on how math is taught, I struggled with it during my freshman year in high school, but then they got a new textbook that had a different system of teaching and I started getting As.