Just adding to starkid's very good explanation: Investing is about personal resources: money, time, energy.
I'm extremely often told things in the vein of your co-worker's comment. People call it "not investing in people" when you allow yourself to be your own person more than you sacrifice for the sake of belonging. This is an Aspie trait, only they think we do it consciously and as a personal choice.
I would've interpreted it as your colleague kind of excusing herself and like saying: "I know you probably don't see it the same way as me, because relationships are not the be-all and end-all for you, but they are for me."
_________________
There are two means of refuge from the miseries of life: music and cats - Albert Schweitzer