Reading the Mind in the Eyes test: Is it really valid?

Page 1 of 2 [ 18 posts ]  Go to page 1, 2  Next

Verdandi
Veteran
Veteran

User avatar

Joined: 7 Dec 2010
Age: 54
Gender: Female
Posts: 12,275
Location: University of California Sunnydale (fictional location - Real location Olympia, WA)

05 Jul 2011, 3:30 pm

swbluto actually found with a somewhat limited sample here that there was no correlation between ability to read the eyes in the test with whether someone was autistic or not.

I found that there is more empirical evidence for this:

http://incorrectpleasures.blogspot.com/ ... t-and.html

Quote:
I plan to write more about this later, but one thing that I can do now after reading the full text of the Austrian study that was published in 2006 that involved the RMET is that the mean score in the RMET for 206 normal male controls was 22.35 and the mean score for 217 normal female controls was 23.31 in that test, both being rather lower scores for normal controls than have been found in many other studies of the RMET. I believe this is because this study is superior, as I will explain later.

So, the figures quoted by Cordelia Fine in her interesting book Delusions of Gender were true, and the claim by Prof. Simon Baron-Cohen in his 2003 book The Essential Difference that a score of 30 in this test is a typical average score looks most questionable in light of the 2006 Austrian study. This discrepancy is especially important when one considers that scores by autistics on this test of around 20 to 22 have been cited as evidence of a deficit in reading emotions or "empathizing" in autistic people. No score on any test can be judged to be superior or normal or subnormal without first having an understanding of what a normal score is, based on solid scientific research studies. So do autistic people really have a deficit in reading human emotions in facial expressions in the eyes in photographs of eyes?


I don't have anything to say about the ability or lack thereof to read emotions in eyes (or facial expressions), but then I think a more valid test using these pictures would be, "What is this person's mood and why?" At which point NTs are likely to produce a social scenario and an autistic person is likely to say "Those eyes look like [emotion]." The relative scores may not be relevant at all.

The blog links this study: http://www.sciencedirect.com/science/ar ... 6906002212

Quote:
Abstract

The second-to-fourth digit ratio (2D:4D) is sexually differentiated and is a likely biomarker for the organisational (permanent) effects of prenatal testosterone on the human brain. Recent research has highlighted a possible role of prenatal testosterone levels in both the etiology of autism-spectrum disorders and in sex and individual differences in cognitive styles of the normal mind (Baron-Cohen’s Extreme Male Brain Theory of Autism and Empathising/Systemising Theory).

Importantly, autistic children present lower (hypermasculinised) 2D:4D than healthy controls. Based on these accounts, we investigated the relation of 2D:4D with Baron-Cohen’s measures of empathising (“Reading the Mind in the Eyes” test, RMET; Empathy Quotient, EQ), systemising (Systemising Quotient, SQ), and autistic-like traits (Autism-Spectrum Quotient, AQ) in the general population (N = 423 Austrian adults). Whereas sex differences into the expected direction and of expected size were obtained for all variables and internal scale consistencies tallied to retrievable reference values, 2D:4D was unrelated to RMET, EQ, SQ, and AQ scores.

Candidate explanations for this lack of correlation might be possible developmental timing differences in the expression of 2D:4D and empathising/systemising, qualitative (as opposed to quantitative) functional differences between the normal and the autistic mind, or the suboptimal psychometric properties of the measures.


And this is the full post she promised in the first post I linked:

http://incorrectpleasures.blogspot.com/ ... t-add.html

Quote:
Things don't add up when we compare the quotes from the two different books. Baron-Cohen was most certainly wrong when he wrote that a typical, normal score on that test is 30. So far I have had a close look at eight different studies that have involved giving a normal control group the same 36-question version of the Reading the Mind in the Eyes Test (RMET), and in all of those studies the normal control group's mean score was under 30. Baron-Cohen even contradicted his own research findings when he claimed in the book published in 2003 that 30 was a normal score on the 36-question RMET. A study co-authored by Prof. Baron-Cohen that was published in 2001 in an autism journal indicated that a normal score on the RMET for “general population controls” is around 26 out of 36. I have not yet been able to get a hold of the full-text of the Austrian study, but if Fine's account of the scores is correct a normal score on the RMET is nowhere near 30.

Should it be this difficult to find out what a normal score on a test is? Of course it shouldn't! A person should be able to trust a professor when he says that you have scored below normal on a test. With a score of 25 out of 36 in the RMET, and a normal score in the RMET being well under 30 and possibly as low as 22 out of 36 for men, it looks like Richard Borcherd's ability to read facial expressions in the eyes is completely normal, maybe even good, and possibly even better than the average woman's score. But Borcherds is autistic, and autistic people are emotion-blind, aren't we? And women have super powers of emotion-reading by virtue of our hormones, don't we? [Why do I feel so confused?] So any woman should be far superior to some autistic male mathematician in a test of identifying facial expressions, surely? Well, maybe not.


The post is much longer than those two paragraphs.

Just to be clear, I don't think she's proposing no autistic person ever has trouble reading emotions from facial expressions. I think she's proposing that it is not necessarily such a strong deficit across the board.



wavefreak58
Veteran
Veteran

User avatar

Joined: 26 Sep 2010
Age: 66
Gender: Male
Posts: 4,419
Location: Western New York

05 Jul 2011, 3:52 pm

Interesting stuff.

Am I seeing a pattern in Baron-Cohen's work? He seem a bit sloppy and subject to a large dose of confirmation bias.

The Minds Eye test is useless to me. I score very high on it, but I also study portraiture and during the test I can take as much time as I want to figure out the correct response. But in real life, I suck at reading emotions.


_________________
When God made me He didn't use a mold. I'm FREEHAND baby!
The road to my hell is paved with your good intentions.


btbnnyr
Veteran
Veteran

User avatar

Joined: 18 May 2011
Gender: Female
Posts: 7,359
Location: Lost Angleles Carmen Santiago

05 Jul 2011, 3:58 pm

I think that the whole test has nothing at all to do with real-time face-to-face interactions during which autistic people experience difficulties. I scored 34/36 on it, because it consisted of photographs of eyes on a computer screen, i.e. objects, and its format was multiple choice, i.e. highly amenable to the intellectual analysis that is my bread and butter, meat and potatoes, etc. And there was no time limit or pressure to respond. In fact, I would expect myself to do well on exactly this type of test.

It is unclear to me whether NTs taking this test are using conscious intellectual analysis or subconscious intuition. These are not people in front of their faces. These are only photos of human eyes with lists of multiple choice words. Are they also using intellectual analysis for the test? They wouldn't be in real life.

A better format for the test would have been fill-in-the-blank. I would have done terribly on such a test. When looking at the pictures, NTs would have been able to think of all kinds of possible emotions, come up with all kinds of stories to account for them, and given complex answers, possibly through intellectual analysis rather than subconscious intuition of the type that happens in real-time face-to-face interactions. My mind would have struggled to come up with any but the most basic answers of "good feeling" or "bad feeling", and I would likely have made a comment about the geometric contours of someone's eyebrows or eyelashes.

I agree with this approach for a more valid test:

Verdandi wrote:
I don't have anything to say about the ability or lack thereof to read emotions in eyes (or facial expressions), but then I think a more valid test using these pictures would be, "What is this person's mood and why?" At which point NTs are likely to produce a social scenario and an autistic person is likely to say "Those eyes look like [emotion]." The relative scores may not be relevant at all.


And yes, the researchers really need to nail down what the normal scores are before they start interpreting the results!



Verdandi
Veteran
Veteran

User avatar

Joined: 7 Dec 2010
Age: 54
Gender: Female
Posts: 12,275
Location: University of California Sunnydale (fictional location - Real location Olympia, WA)

05 Jul 2011, 4:16 pm

Yeah, I agree with both of you - interpreting these things in real time is very different from interpreting them from a multiple choice test with all the time you need.

I also agree that there is a lot of confirmation bias in Baron-Cohen's work. You can also tell when his objection to criticism from autistic people is to say "the people criticizing me are not autistic enough to experience what I'm talking about."



marshall
Veteran
Veteran

User avatar

Joined: 14 Apr 2007
Gender: Male
Posts: 10,752
Location: Turkey

05 Jul 2011, 4:33 pm

In general no. Correlation != Causation. There is a wide statistical spread to the scores in both NT and autistic populations compared to the difference in the means of the two populations. That pretty much rules out emotion reading being the core "causative" agent in autism.

Differences in interhemispheric brain connectivity / white-matter is a more biologically founded causative basis for autistic deficits. Impairment tends to occur wherever too many physically separated areas of the brain are required to function and communicate with each other simultaneously. Language and social interaction is just one aspect. Sensory and emotional integration is another.



draelynn
Veteran
Veteran

User avatar

Joined: 24 Jan 2011
Age: 55
Gender: Female
Posts: 2,304
Location: SE Pennsylvania

05 Jul 2011, 4:36 pm

I agree with the wavefreak.

Determining emotion from a static picture is a much different animal than in the moment, in front of someone. You can focus on a single task where, in person, I think many of us are concentrating on a plethora of different things at once. I can look at pictures of eyes. I can't look a live person in the eyes long enough to bother with expression. I rely heavily on tone of voice and mouth expressions. Yet, thanks to my art education, I can interpret eye expressions fairly well from pictures.

And, I can make up long, detailed, emotion packed stories about people just from their expression as well - on the spot, without notice. In many circles that would preclude an ASD dx altogether. I'm not sure how active daydreamers would get pegged with an unimaginative label but, it too persists.



Verdandi
Veteran
Veteran

User avatar

Joined: 7 Dec 2010
Age: 54
Gender: Female
Posts: 12,275
Location: University of California Sunnydale (fictional location - Real location Olympia, WA)

08 Jul 2011, 1:57 pm

draelynn wrote:
I agree with the wavefreak.

Determining emotion from a static picture is a much different animal than in the moment, in front of someone. You can focus on a single task where, in person, I think many of us are concentrating on a plethora of different things at once. I can look at pictures of eyes. I can't look a live person in the eyes long enough to bother with expression. I rely heavily on tone of voice and mouth expressions. Yet, thanks to my art education, I can interpret eye expressions fairly well from pictures.

And, I can make up long, detailed, emotion packed stories about people just from their expression as well - on the spot, without notice. In many circles that would preclude an ASD dx altogether. I'm not sure how active daydreamers would get pegged with an unimaginative label but, it too persists.


I meant to reply to this:

I think that being unable to do that is indicative, but not so much that being able to do that means you're not. They test so many things, though, that it's just one of many.

There was actually a study that claimed autistic people can't daydream at all, too. Clearly, it's nonsense, but here:

http://www.nature.com/news/2006/060508/ ... 508-3.html

I also want to say that I am sick of the word "healthy."



merrymadscientist
Veteran
Veteran

User avatar

Joined: 18 Dec 2007
Age: 46
Gender: Female
Posts: 533
Location: UK

08 Jul 2011, 3:01 pm

My general feeling from experience and having read Cordelia Fine's book is that Simon Baron-Cohens conclusions are not right. The problem with psychology is that it is so open to multiple interpretations, and Fine's idea that women tend to rate themselves as more empathetic because they think they ought to be is something I can easily see happening.

My general impression is that NT men and women are equally good at reading social cues, and for myself I am also pretty good at reading them IF I notice (I often don't as I don't do much eye contact), but I have no idea how to respond. So I can be empathetic but I can't express it. I really dislike the hyper-masculinisation theory of autism - yes I am not very feminine, but I am also not very masculine and my strong feeling is that gender is almost completely social, but having not picked up on social cues quite so well as a child through not looking at people as much (and having over analysed those I realised were there and found most of them to be idiotic) I have not become excessively feminine. I do actually think that autistics are probably better at systemising than NTs in general, however I do not think that this is a masculine trait - rather a trait that has stereotypically assumed to be masculine.



Janissy
Veteran
Veteran

User avatar

Joined: 5 May 2009
Age: 57
Gender: Female
Posts: 6,450
Location: x

08 Jul 2011, 3:19 pm

wavefreak58 wrote:
Interesting stuff.

Am I seeing a pattern in Baron-Cohen's work? He seem a bit sloppy and subject to a large dose of confirmation bias.

The Minds Eye test is useless to me. I score very high on it, but I also study portraiture and during the test I can take as much time as I want to figure out the correct response. But in real life, I suck at reading emotions.


I think it's a real weakness in the test that it doesn't mimic real life situations. It is static pictures of eyes only and not only that but it is pictures of actors and models who are not displaying their actual emotions but what a director told them to display. So it's also a test of how well they can act.

One of the times that this test was brought up, another poster and I recognized one of the sets of eyes as being the model Claudia Schiffer.
Here is the test:

http://glennrowe.net/BaronCohen/Faces/EyesTest.aspx


Claudia Schiffer is #34. You have choices such as "terrified" and "aghast". The thing is, in the original ad (which I now can't find on google) she is none of the choices they give. She has the deliberately vague look that all clothing models have in ads so that customers can project their own fantasy feelings onto the model. I don't know where Baron-Cohen got his feelings choices from, but in the ad he took the still from she is merely modeling.

As to the accuracy of the others, I guess you'd have to go through and identify which movie or ad campaign they were lifted from and see if what he gives as the "correct" response is what the actor/actress/model is trying to convey in that scene. That's a job for some cinephile. I recognized Claudia Schiffer only because I wanted to look like her when the ad ran (many, many years ago) and I used to stare at the picture wishing I looked like her- which is the intention of the ad.

At some other point in time, somebody posted a far better test which was of live video. I can't remember if the people were acting (which can distort the accuracy, depending on their acting skill) or if they were filmed thinking they were taking part in something else and then their emotions were elicited...somehow. Anyway, it was live video so it was more realistic.



ocdgirl123
Veteran
Veteran

User avatar

Joined: 10 Oct 2010
Age: 28
Gender: Female
Posts: 2,809
Location: Canada

08 Jul 2011, 5:52 pm

I do fine with facial expressions in real life, but really bad on the mind in the eyes test. I think it's because they show just the eyes and not the whole face.



Amajanshi
Veteran
Veteran

User avatar

Joined: 1 Apr 2009
Age: 36
Gender: Male
Posts: 626

08 Jul 2011, 10:45 pm

Interesting studies. I recall when I did the "RTME" test as part of my AS assessment, I failed and got 18/40. Since the diagnosis, I practiced looking at facial expressions and have improved in my ability to recognize the *static* versions. However, I still have trouble reading facial expressions and assessing other channels of communication (besides verbal) on the spot in a conversation as I'm automatically focusing far more on the words.



Gwenwyn
Blue Jay
Blue Jay

User avatar

Joined: 2 Jul 2011
Gender: Female
Posts: 94

08 Jul 2011, 11:02 pm

The 22-24 version sounds about right. I posted this on facebook (oh yes, a totally random sample, right?) and most of my friends scored 22-24, and one got 30. I got 14 -_-' Better than guessing randomly, but not by much.



Verdandi
Veteran
Veteran

User avatar

Joined: 7 Dec 2010
Age: 54
Gender: Female
Posts: 12,275
Location: University of California Sunnydale (fictional location - Real location Olympia, WA)

08 Jul 2011, 11:37 pm

Amajanshi wrote:
Interesting studies. I recall when I did the "RTME" test as part of my AS assessment, I failed and got 18/40. Since the diagnosis, I practiced looking at facial expressions and have improved in my ability to recognize the *static* versions. However, I still have trouble reading facial expressions and assessing other channels of communication (besides verbal) on the spot in a conversation as I'm automatically focusing far more on the words.


Yeah, I have tons of issues in real time that don't happen in a test. I'm somewhat better at watching other people while not involved than I am while involved. Or perhaps that only works for television comedies where everyone has exaggerated expressions.

My first score on it was 19, but I've taken it so many times I get about 22-25. I got 30 once when I chose the "seductive/sexy" answers for all the female pictures. I'm not sure how many of those were accurate, though, but it does erode my faith in the test.



Amajanshi
Veteran
Veteran

User avatar

Joined: 1 Apr 2009
Age: 36
Gender: Male
Posts: 626

09 Jul 2011, 12:09 am

Verdandi wrote:
Yeah, I have tons of issues in real time that don't happen in a test. I'm somewhat better at watching other people while not involved than I am while involved. Or perhaps that only works for television comedies where everyone has exaggerated expressions.

My first score on it was 19, but I've taken it so many times I get about 22-25. I got 30 once when I chose the "seductive/sexy" answers for all the female pictures. I'm not sure how many of those were accurate, though, but it does erode my faith in the test.


If they made the test harder and didn't make it multiple choice, and requiring you to state the emotion that comes into your mind when you see the eyes, then I think the discrepancy between ASD and NT individuals would be more significant.

There may also be collusions with Alexithymia (difficulty understanding emotions) which is more common in ASD people however, so that'd have to be taken into consideration as well.

I remember in the RTME test, there were words that even I couldn't understand, so I chose the words which I got, providing they were still an approximation of what I thought the eyes meant.



Verdandi
Veteran
Veteran

User avatar

Joined: 7 Dec 2010
Age: 54
Gender: Female
Posts: 12,275
Location: University of California Sunnydale (fictional location - Real location Olympia, WA)

09 Jul 2011, 12:11 am

Amajanshi wrote:
If they made the test harder and didn't make it multiple choice, and requiring you to state the emotion that comes into your mind when you see the eyes, then I think the discrepancy between ASD and NT individuals would be more significant.

There may also be collusions with Alexithymia (difficulty understanding emotions) which is more common in ASD people however, so that'd have to be taken into consideration as well.

I remember in the RTME test, there were words that even I couldn't understand, so I chose the words which I got, providing they were still an approximation of what I thought the eyes meant.


I agree about making the test harder. I think Baron-Cohen's screening tools are actually kind of subpar.

I also think that's a good point about alexithymia.



League_Girl
Veteran
Veteran

User avatar

Joined: 4 Feb 2010
Gender: Female
Posts: 27,205
Location: Pacific Northwest

09 Jul 2011, 12:14 am

That online test is nothing but just a guessing game. You look at the eyes and then look at the answers and you pick one of them you think is correct. But you have to make a guess. Plus you can keep going back and take it again and again until you get a high score. :wink: