Anyone good with probabilities (Bayes Theorem)

Post Reply New Topic

There is a paper "Finding the True Number of Females with Autistic Spectrum Disorder by Estimating the Biases in Initial Recognition and Clinical Diagnosis" (https://www.mdpi.com/2227-9067/9/2/272) that is using Bayes Theorem to derive the prevalence of autism in women.

In the paper, the author uses the prevalence of autism in women with borderline personality disorder and the prevalence of borderline personality disorder in women with autism to derive the prevalence of autism in women, using the following equation:

P(ASD|BPD) × P(BPD) = P(BPD|ASD) × P(ASD)

After plugging in the numbers, the author gets:

0.146 × 0.062 = 0.15 × P(ASD)
P(ASD) = 0.060

However, if you look at the sources cited, the equation is more like:

P(ASD|BPD resistant to normal treatment and referred for MBT) * P(BPD) = P(BPD|ASD|Normal IQ|Adults with childhood onset neuropsychiatric disability) * P(ASD)

As far as I know, generalizing a subset of a population as representative of the whole population will result in inaccuracies. However, the two peer reviewers for this paper did not say anything about this and this paper has even been cited by 2 more papers that are published in the journal Autism In Adulthood.

Is this paper junk science or am I missing something?

So here are my qualifications: PhD in Applied Econ, took a ton of statistics in grad school, and work in data science.

My initial inclination is that if its published it's "acceptable." However, I should note that typically someone will do a "meta" study where they look through multiple studies and then try to get an average from that. Any one study can be wrong so the thinking is if you combined them and weight the results somehow, you can get a better result.

I do not know enough about their data to make any conclusion. In general, psych studies have a replication problem in that people can't replicate results. Medical studies in particular tend to have low sample sizes.

I opened this one up and started reading and immediately I was concerned by this line below Figure 1:

"All subjects were either diagnosed by me or, if already diagnosed when referred for management, were assessed by me as meeting DSM-5 criteria." They did say this later on, "My practice was to make the diagnosis myself if sure but refer the younger children to a psychologist for assessment if unsure."

So immediately thats a huge flag for two reasons. The writing should have stated "by the author" as personal pronouns are generally not used. But more concerning, one person diagnosing is going to introduce a bias right there.

Even if the math is correct then, the estimates will be off depending on how much bias the author has in diagnosing. If its zero, then its not off. But the odds of it being zero are pretty low as everyone has a bias. There's also that line about the younger children - that will bias the age if the unsure diagnoses have a different bias than the author.

Overall I did not see any methodological or mathematical errors. I would just say that a better estimate would be to have lots of people repeat this on different samples and take averages. There is no control for the bias of the people diagnosing. I found it concerning that no confidence intervals were reported. However, to get confidence intervals for that they would have to bootstrap it via computer simulation.

Even if the people diagnosing have no "bias" compared to each other, there could be a bias in Australia vs rest of world. Maybe the government there passed some extra funding for kids diagnosed so they are over diagnosing. Or maybe only richer parents send their kids to get evaluated. Who knows. I don't.

The only way to truly find out would be to take repeated samples of 1,000 women and assess them. This would be expensive so researches try to find tricks to make estimates based on cheaper data.

Fundamentally, I don't see any reason why there should be much of a difference in autism prevalence rates in males and females. But again, I don't know that much about that stuff. Still, their estimate seems plausible and is a probably within an order of magnitude of the actual number.

Isn't assuming P(ASD|BPD resistant to normal treatment and referred for MBT) is equal to P(ASD|BPD) a mathematical error or methodological error? Or is it a logical error?

For example, in his equation 0.146 × 0.062 = 0.15 × P(ASD), the 0.146 is from https://www.researchgate.net/profile/Goeran-Ryden-2/publication/228478050_Borderline_personality_disorder_and_Autism_Spectrum_Disorder_in_females_-_A_cross-sectional_study/links/0c960519dc1328610a000000/Borderline-personality-disorder-and-Autism-Spectrum-Disorder-in-females-A-cross-sectional-study.pdf.

If you read the Population section under Materials and Methods, it is pretty clear that the sample used for the 14.6% figure is not in anyway representative of women with BPD at all.