Sloppy Biomedical Research a problem in USA

Post Reply New Topic

Biomedical science faces an existential threat — not the threat of budget cuts or anti-science political views, but one from within the discipline itself. So argues Richard Harris in his new book, “Rigor Mortis.”

The book is an alarming and highly readable summation of what has been called the ‘reproducibility crisis’ in science — the growing realization that large swathes of the biomedical literature may be wrong. By one estimate, according to reporting by Harris, up to 85 percent of published studies may be incorrect. By another, unreliable research costs U.S. taxpayers $28 billion per year — an amount equivalent to nearly the entire annual budget of the U.S. National Institutes of Health (NIH)

Most worryingly, Harris reports, an untold number of people have participated in drug trials on the basis of demonstrably false findings, putting their own lives at risk.

He highlights an analysis of nine drugs that showed promise for treating amyotrophic lateral sclerosis (ALS) in animal studies. Eight of the drugs failed to slow the deadly neurodegenerative disease in people.

When scientists at the ALS Therapy Development Institute in Cambridge, Massachusetts, explored why the trials were a bust, the problem turned out to be worse than the known issue that animal models don’t faithfully reproduce human ALS. The scientists found a host of flaws in the animal studies themselves. Some studies included too few animals to detect a statistically valid effect; others failed to account for gender differences. Male mice in the ALS model die earlier than females, and in some of the studies, researchers included more females in the treatment group than in the control group — which could make the drug look like it’s working even if it isn’t.

When the Cambridge scientists redid the studies with more rigorous methods, not one of the candidate drugs showed a benefit. The clinical trials had been destined to fail, at a cost of hundreds of millions of dollars.

Most scientists don’t intentionally conduct shoddy experiments, Harris notes. They want to produce findings that benefit society. But science has evolved in ways that make it difficult for them to follow that dream. Funding is scarce and the pressure to publish in high-impact journals is intense, he writes. As a result, scientists prioritize studies that are likely to make a splash over less sexy but perhaps more rigorous work.

Through interviews with researchers and journal editors, Harris learns that the pressure to publish and secure precious grant money drives scientists to engage in behaviors that undermine the reliability of their work. These include ignoring data that don’t fit with their theories, massaging statistics and overhyping results. Some scientists fabricate data outright.

Once a flawed study is published, there is no incentive for other scientists to attempt to replicate its results or point out its flaws, because they must spend their time and money developing and publishing their own research if they wish to advance their careers. And there is no way for scientists to admit their mistakes without damaging their reputations.