Study finds huge flaws in media articles on research topics

Post Reply New Topic

My heart sinks every time I see some sloppy study making unfounded assertions about autism, and sinks even more when I see it used by AS people and others as if the claims were factual and indisputable. Science is a process, not a set of single studies.

The journalist's resource organisation has been concerned about health studies reporting generally, and has some wise words on the issue:

https://journalistsresource.org/studies ... self"

Depressing, but I won't pretend I'm surprised.

Thanks for the link, B19; you always find the good stuff!
Timely too; I was discussing a very similar issue elsewhere yesterday, and this adds some substance to the arguments I was making. This is one of the reason's that I hate to see papers hidden behind paywalls. Merely being able to point out that some phrasing in an article is hyperbolic, or that a statistical technique is susceptible to bias in general, is no substitute for concrete comparisons between an article and its sources, and is much easier to wave away as linguistic or mathematical pedantry.

That's a very revealing study.

The following findings made last month could supplement its cautions:
400,000 Scientists All Over the World Have Been Published in Fake Journals - Journalists investigated widespread fraud within the scientific publishing community.

^ Psuedo-science publishing has been a theme in the back pages of New Scientist just lately. It's a serious issue of course, but they've been highlighting some of the ridiculous "trolling" papers that some scientists have been submitting to see what nonsense they're prepared to take cash for. For example; Solicitation of patient consent for bilateral orchiectomy in male canids by Two Dogs and A Chicken (the medical jargon means; "Asking for dogs' consent before removing their testicles".)

In the past five years - since Nature and other more ethical journals exposed the "peer reviewing scandal" - there has been more comment and more awareness however changes for the better have been very focused it seems, rather than widespread, and few countries or journals seem to have made an apparent and comprehensive effort to stop the rot. The UK seems to be doing the most and I'm unaware of any similar response in the USA, though perhaps I missed it...

The tabloid level of media reporting has fallen even further during that period in terms of unfounded and sensationalist claims. The Daily Mail specialises in this in the UK it seems and it is viral across the USA media generally, much more so.

In 2007, two scientists (surnames Triggle and Triggle, which for some reason makes me think of Alice in Wonderland!) published a comprehensive exposition of their concern about the frauds apparent to them 11 years ago. A quote from that paper:

Scientific fraud is now beginning to be seen as no different from any other criminal and often perpetrated by a repeat offender. A US-based survey suggests that the incidence of falsification, fabrication and plagiarism is higher than one would have hoped with approximately 33% of the participants admitting to one or more of the top 10 (mis) behaviours (Martinson et al 2005). So where should behaviour modification begin? Presumably such modification should start at the top with national governments and academic institutions establishing policies that are both followed and enforced.

Indeed, fraud in science, whether initially intended as hoaxes or planned with career and profit-making intentions, not only ruins the careers of the perpetrator, but also, potentially, their innocent colleagues, as well as tarnishing the reputation of the institution where the work was performed and reducing the confidence of the public in the value of scientific research. Fraud in health research may also have direct or indirect negative effects on health care and peoples lives; fraud in other areas of science may, of course, affect the economy and the lives of people. It can been argued that one approach to dealing with scientific fraud is to proceed through civil courts as, likely, misuse of grant funds is also involved (see Smith 2005).

It seems to me that things are in a worse state now, and recourse to civil remedies may be the only way of stemming some of the more egregrious misclaims. This already happens in the matter of the theft of intellectual property in copyright law, however that has been applied in commercial settings, not academia - though it could be used to stop the rot in academic publishing.

"Study finds huge flaws in media articles on research topics." I say that more study is needed.

Yes, it is, and we need better training in scientific literacy generally. However all the training and more studies are not going to solve the lack of will to implement remedies. That's where the main barrier to cleaning house is.

Individual fraudsters are occasionally made an example of, however that acts as a smokescreen for the thousands who continue to do the same and worse and laugh all the way to grants and the bank.

It spreads like the flu. The virus of one under-the-radar fraudster in science infects the students under him/her, colleagues, institutional boards and so on.

Just a guess, but maybe most of the problem arises from The Media having to "dumb down" the language of scientific research for the general population, thus losing the real meaning in a flood of colloquialisms, flawed language, and journalistic embellishments.

Those research papers can be very math-intensive and use some very technical language that only another researcher in the same field can truly understand.

Translating the essence to simple language without compromising truth is needed, and I agree that journalists so lack scientific training, statistical competence and awareness of how fraud is accomplished that they are incapable of weeding out their own contribution to the misrepresentations.

That onus should fall on the institutions where studies are approved and given permission. There needs to be a post approval review process and tracking from that point by specially appointed academics, fluent in critical thinking, statistical skulduggery, philosophy of science and their own particular discipline eg psychology.

This could happen. Such people already exist, perhaps in fewer number now that academia has become more commercial.

Disclaimer: Whether or not the following story represents a continuous chain of events is entirely speculative.

Take the concept of "Negative Differential Resistance" (NDR), where a component responds to an increase in voltage across it by decreasing the current through it, and vice-versa. I once showed a graph of this phenomenon to a journalism student, described the process in the simplest terms, and even gave a small practical demonstration where I turned up the voltage and watched an incandescent lamp get dimmer. The journalism student seemed fascinated, as she took copious notes.



The next week, I'm reading an article in the school newspaper about how I had discovered a source of energy heretofore unknown by modern science. The physics profs wondered what I was doing, so I gave them the same demonstration, which satisfied them that I wasn't some crackpot working in their midst.

The journalism student was chided for the 'license' that she took on the truth, but she insisted that I had discovered something that was now being suppressed by the 'establishment'. You can now find conspiracy theorists and fringe 'scientists' declaring that NDR represents a key component in secret government projects involving zero-point energy, anti-gravity, and death rays.

I strongly agree that there needs to be more scientific literacy at all levels of society. Unfortunately, that likely will not happen in my lifetime. That is something that I will just have to learn to live and let be. It is irritating to me that people have an aversion to wanting to learn something new if it takes effort. I will stop my rant here on that topic.



As for the fraudulent scientific journals, I received emails often from them trying to get me to purchase article space in their magazine. For $1000 to $3500, they will publish literally anything that you like. Those emails go right to my spam file folder and are soon deleted. I really hate those as they are trying to circumvent the scientific peer review system just to make a buck.

Yes, there are those who will feel cornered in their jobs that fall for their tricks. Many tenure-track teaching positions require proof of being current in their subject matter by publishing novel research articles in the scientific literature. But, it can be very hard to get something published in a peer reviewed journal if you do not have a large name on the article that they know. Years ago, I tried to publish some of my quantum physics research in two well-respected journals but got rejected by them. One of the journals really liked my ideas, but they just did not know who I was and did not want to risk publishing it due to that fact. (I will be honest that it did hurt to get that rejection.) The other was just not interested in publishing in that area of theoretical physics, as they are heavy into experimental physics.

So, some take the easy way out and pay to have the work published, rather than to go through the long process of getting it published the right way. All they are concerned is to put it into a form that they can claim is published. They then have the "proof" that they are current in their area and can possibly become tenured (if they finish the rest of the process). However, many universities are starting to look into what and where the research is published as they should. At those institutions, if it is not a peered-reviewed journal, it actually counts against the person. As they say in the academic world, it is publish or perish. But, it also needs to be done the right way.



To be clear, I am in a non-tenure track position, so that issue does not apply to me yet. I have been authorized to be able to do laboratory research now, but I have no job requirement to do so. There are some ideas that I am exploring to create a new novel material for a very specific purpose. I would have to move up the ladder to a tenure-track position (not likely to happen here) before I would have to worry about how much I publish per year and where. I have been published this year in an chemical educational article that I helped with, so that would count in my total. Same with the book chapter that I have been asked to help write by next year.

Oh, great more stupidity and crap which has not been given rigorous critical analysis to determine if something is of great merit or mere garbage. Just like these brainless, cult-like idiots within the continental US whom think that some mythical sky deity created the world and all of it's contents in 6 actual days as, they go by words in a book and that's it or, the equally stupidtrons whom think there was some global flood when, in fact no scientific evidence exists to prove any of this..

Does Google Actually Make Us Dumber? That Study — And Many Others — Were Just Called Into Question. - “Our study in hindsight was outright silly,” one scientist said.