In my experiences, I've noticed when two people are very engaged in a conversation, frequent interruptions of each other is not considered rude or inappropriate.
It seems as an indicator that two people are so engaged and excited by each others conversation, that they cannot keep their racing mouths still and love to jump from topic to topic.
I've also noticed their conversation still strangely flows, even if it is spontaneous and fast-paced.
So, really, it depends on what the conversation was about and how enthusiastic the other person was.
Was it a one-on-one conversation, and were you both enthusiastic/excited to speak together, or did you speak over someone before they got the chance to and it was in a small group?