Yes, I see music as colors and patterns. When improvising I sometimes approach it from a visual sense, "What color would look good here?", sort of abstract expressionist, instead of "what note or chord would sound good here?" I play in nonstandard tunings fairly often and due to that, I hear music in relative pitch so I do not assign colors to given notes. I do see individual colors and colored patterns associated with given intervals however, but they are different for every piece of music. Also when I get my guitar & amp sounding right and the notes "bloom" the way I want them to, I close my eyes to see the colors created. It's sort of like dipping a heavily-charged watercolor brush into a wet-washed field.
I have unconsciously developed a mnemonic scheme to memorize specific parts of the songs I learn - what part I need to play where, etc - by seeing a "preview" of a visual pattern which cues me to insert the musical phrase where it belongs. These patterns are different for every phrase and for every song, though I notice there is a recurrence of an almost houndstooth-like field in a few different songs. The colors are different for each, though.