I have always had this problem. I get very frustrated when people ask me to explain things or elaborate on things, because I have a complete understanding of the answer in my head, but am unable to verbalise it.
I am able to write it down, however. I don't know if this is just because in writing I am able to take my time to explain if necessary, but I find written language far easier than spoken language.
I like the idea of English being a second language, Logicalmom. My first language is visual-tactile, too!