I'm only offended by this type of question if it seems like someone is trying to size me up or determine if it gets their 'okay'. If it's just a casual conversation question and I don't sense you care either way, it doesn't really bother me too much.