Peer-review and publication does not guarantee reliable info

Page 2 of 3 [ 38 posts ]  Go to page Previous  1, 2, 3  Next

kraftiekortie
Veteran
Veteran

Joined: 4 Feb 2014
Gender: Male
Posts: 87,510
Location: Queens, NYC

24 Mar 2019, 5:03 pm

Peer review is not iron-clad—but it’s better than non-peer reviewed.



Pepe
Veteran
Veteran

Joined: 11 Jun 2013
Gender: Non-binary
Posts: 26,635
Location: Australia

24 Mar 2019, 5:07 pm

cyberdad wrote:
They use various tricks to ignore papers ranging from lack of English translation to the topic not aligning with the journal's research focus (whatever they decide is their favourite pet subject).


Interesting...



Pepe
Veteran
Veteran

Joined: 11 Jun 2013
Gender: Non-binary
Posts: 26,635
Location: Australia

24 Mar 2019, 5:18 pm

kraftiekortie wrote:
Peer review is not iron-clad—but it’s better than non-peer reviewed.

According to some, this is a better approach:
Quote:
In addition, it has been shown that large numbers of public reviews are more thorough in reviewing academic articles than a small group of experts [7]. https://www.students4bestevidence.net/p ... formation/



Pepe
Veteran
Veteran

Joined: 11 Jun 2013
Gender: Non-binary
Posts: 26,635
Location: Australia

24 Mar 2019, 5:28 pm

shlaifu wrote:
What amuses me is that the humanities were well aware of that, decades ago. But the "hard" sciences refused sociologists' comments on institutionalized biases. Then the science wars happened, and there's still scientists occasionally publishing joke papers under fake names in humanities journals - to discredit the humanities.
But slowly, the STEM fields are acknowledging that, indeed, the humanities saw this coming.


People who understand human psychology and the primitive base much of it is derived from "shouldn't" be surprised by
the influences of unconscious bias, tribalistic tendencies, tendencies towards corruption and general self-interest...



Pepe
Veteran
Veteran

Joined: 11 Jun 2013
Gender: Non-binary
Posts: 26,635
Location: Australia

24 Mar 2019, 5:44 pm

Quote:
Bias and conflicts of interest among peer reviewers
Pharmacy professor Lisa Bero (2017 Bero, L. (2017). Addressing bias and conflict of interest among biomedical researchers. Journal of the American Medical Association, 317(17), 1723–1724. doi:10.1001/jama.2017.3854
[Crossref], [Web of Science ®], , [Google Scholar]
) reminds that all researchers have personal beliefs, preferred theoretical approaches, and potential conflicts of interest, such as ties to commercial entities. When called upon to review the work of other scholars, it is logical that bias could influence their evaluations. https://www.tandfonline.com/doi/full/10 ... 18.1431443



Pepe
Veteran
Veteran

Joined: 11 Jun 2013
Gender: Non-binary
Posts: 26,635
Location: Australia

24 Mar 2019, 5:52 pm

Quote:
Peer review scams
A common practice by many scholarly journals is allowing authors to suggest potential reviewers —making it incredibly easy to game the system and acquire prompt and glowing reviews. This practice has permitted unscrupulous authors to suggest their own relatives, friends, and colleagues and/or to provide bogus email addresses that are not detected by today's automated manuscript management systems.
https://www.tandfonline.com/doi/full/10 ... 18.1431443



Antrax
Veteran
Veteran

User avatar

Joined: 23 Feb 2019
Gender: Male
Posts: 1,639
Location: west coast

24 Mar 2019, 7:31 pm

Pepe wrote:
Quote:
Peer review scams
A common practice by many scholarly journals is allowing authors to suggest potential reviewers —making it incredibly easy to game the system and acquire prompt and glowing reviews. This practice has permitted unscrupulous authors to suggest their own relatives, friends, and colleagues and/or to provide bogus email addresses that are not detected by today's automated manuscript management systems.
https://www.tandfonline.com/doi/full/10 ... 18.1431443


Suggesting bogus authors is grounds for ethical violations and will cost you your job if you get found out. Any journal worth its salt will make sure the authors you suggest are real people in their established fields. That's another issue is junk journals publishing literally anything that crosses their desks. Bad science sometimes goes through the reputable journals, but it is much less frequent.

I think the best way to handle peer review is to acknowledge it is imperfect, and use your own thinking when it comes to reading scientific articles. That said you'll want to have some understanding of the subject before critiquing it. I can comfortably understand synthetic biology papers methods and conclusions, but would be hopeless trying to critique a physical chemistry paper.


_________________
"Ignorance may be bliss, but knowledge is power."


cyberdad
Veteran
Veteran

User avatar

Joined: 21 Feb 2011
Age: 58
Gender: Male
Posts: 36,036

25 Mar 2019, 2:31 am

kraftiekortie wrote:
Peer review is not iron-clad—but it’s better than non-peer reviewed.


And that's really about the gist of it...



Pepe
Veteran
Veteran

Joined: 11 Jun 2013
Gender: Non-binary
Posts: 26,635
Location: Australia

25 Mar 2019, 4:47 am

Antrax wrote:
I think the best way to handle peer review is to acknowledge it is imperfect, and use your own thinking when it comes to reading scientific articles.


Agreed...
But there are people who are so tunnel visioned they can't accept even the possibility...



Darmok
Veteran
Veteran

User avatar

Joined: 18 Dec 2015
Gender: Male
Posts: 12,030
Location: New England

03 Apr 2019, 1:55 pm

This went on for twelve years.

Duke University to pay $112.5 million for faking scientific research data

Duke University has agreed to pay $112.5 million to the US government for scientific research misconduct, the Department of Justice announced in a press release on Monday. The fine comes as a result of the university falsifying research on federal grants.

The Department of Justice lays out the background and extent of the fraud. Duke received millions of dollars from the National Institutes of Health and the Environmental Protection Agency every year, but it turns out that the university engaged in over two dozen acts of fraud against the government.

“The settlement resolves allegations that between 2006 and 2018, Duke knowingly submitted and caused to be submitted claims to the NIH and to the EPA that contained falsified or fabricated data or statements in thirty (30) grants, causing the NIH and EPA to pay out grants funds they otherwise would not have,” the release states.


https://www.thecollegefix.com/duke-univ ... arch-data/


_________________
 
There Are Four Lights!


Fnord
Veteran
Veteran

User avatar

Joined: 6 May 2008
Gender: Male
Posts: 60,949
Location:      

03 Apr 2019, 2:02 pm

Darmok wrote:
This went on for twelve years. Duke University to pay $112.5 million for faking scientific research data ...
This also proves that The System works -- Duke University was found out, investigated, and fined for their efforts -- even if The System works slowly.

$112.5 million is likely to be more than what the U.S. government paid to Duke in research grants, by the way.



Pepe
Veteran
Veteran

Joined: 11 Jun 2013
Gender: Non-binary
Posts: 26,635
Location: Australia

03 Apr 2019, 4:21 pm

Darmok wrote:
This went on for twelve years.

Duke University to pay $112.5 million for faking scientific research data

Duke University has agreed to pay $112.5 million to the US government for scientific research misconduct, the Department of Justice announced in a press release on Monday. The fine comes as a result of the university falsifying research on federal grants.

The Department of Justice lays out the background and extent of the fraud. Duke received millions of dollars from the National Institutes of Health and the Environmental Protection Agency every year, but it turns out that the university engaged in over two dozen acts of fraud against the government.

“The settlement resolves allegations that between 2006 and 2018, Duke knowingly submitted and caused to be submitted claims to the NIH and to the EPA that contained falsified or fabricated data or statements in thirty (30) grants, causing the NIH and EPA to pay out grants funds they otherwise would not have,” the release states.


https://www.thecollegefix.com/duke-univ ... arch-data/


People work the system...
Yep...



Antrax
Veteran
Veteran

User avatar

Joined: 23 Feb 2019
Gender: Male
Posts: 1,639
Location: west coast

03 Apr 2019, 7:46 pm

Fnord wrote:
Darmok wrote:
This went on for twelve years. Duke University to pay $112.5 million for faking scientific research data ...
This also proves that The System works -- Duke University was found out, investigated, and fined for their efforts -- even if The System works slowly.

$112.5 million is likely to be more than what the U.S. government paid to Duke in research grants, by the way.


Agreed. Punishment will be severe, scientists for all their faults take fake data very seriously.


_________________
"Ignorance may be bliss, but knowledge is power."


cyberdad
Veteran
Veteran

User avatar

Joined: 21 Feb 2011
Age: 58
Gender: Male
Posts: 36,036

04 Apr 2019, 12:50 am

The Duke University story has nothing to do with peer review or indeed the integrity of Duke or it's academic staff...

According to the British Medical journal (BMJ) The case refers specifically to 2013 to Erin Potts-Kant, a clinical research coordinator at the Foster Lab which is "loosely affiliated" with Duke. Potts-Kant conducted mouse lung tissue experiments for many Duke departments and external institutes.

She was arrested for spending the laboratory’s research budget on internet purchases, which triggered an investigation by Duke into her research. Subsequent analysis of her data and attempts to reproduce her results found that none of her work dating back to 2006 could be relied on.

Note: the Foster lab was not the recipient of the grants, they were essentially sub-contracted by Duke academic staff to run the trials.

The issue here is the individual in question was working for her own personal benefit and commercial advantage by fabricating lung tissue contamination data to make it look like environmental pollution issues were worse than they actually were so that more money could be funnelled into grants keeping her in business.

Most major universities rely on external laboratories and pathology labs to collect data for them. It represents an element of risk in terms of the control accurate sample analysis being placed in the hands of an outside party...but has nothing to do with peer review



SocOfAutism
Veteran
Veteran

Joined: 2 Mar 2015
Gender: Female
Posts: 3,078

04 Apr 2019, 9:00 am

The thing is, this system could easily be improved.

When you’re doing math, you go back over your work by checking it via a different method- do it backwards, use a calculator, whatever. An experiment can be checked by being replicated by changing everything except whatever it is you’re actively testing. Then you can be pretty sure of your assumption.

A good example of this is Milgram’s experiment where participants follow instructions given by a man in a lab coat to pull a lever or push a button. The participants think they are inflicting pain on another person with the lever or button. This experiment was famously replicated with many, many different conditions. So we can confidently say that people do follow instructions from a perceived authority figure, even if they think it is hurting another person. Disturbing, and QUITE useful to know about humans.

Boy, wouldn’t it be great to know more things with such confidence?

BTW...Milgram’s experiment was a social psychology experiment. Social psychology can be sociology or psychology depending on how you look at it. This kind of experiment is no longer deemed ethical, which is why we only get observational or voluntary data now. I was taught that the kind of study that sociologists CAN do now will never be as “good”, data-wise, as they were back when they were, ahem, doing unethical things.



Antrax
Veteran
Veteran

User avatar

Joined: 23 Feb 2019
Gender: Male
Posts: 1,639
Location: west coast

04 Apr 2019, 5:21 pm

SocOfAutism wrote:
The thing is, this system could easily be improved.

When you’re doing math, you go back over your work by checking it via a different method- do it backwards, use a calculator, whatever. An experiment can be checked by being replicated by changing everything except whatever it is you’re actively testing. Then you can be pretty sure of your assumption.

A good example of this is Milgram’s experiment where participants follow instructions given by a man in a lab coat to pull a lever or push a button. The participants think they are inflicting pain on another person with the lever or button. This experiment was famously replicated with many, many different conditions. So we can confidently say that people do follow instructions from a perceived authority figure, even if they think it is hurting another person. Disturbing, and QUITE useful to know about humans.

Boy, wouldn’t it be great to know more things with such confidence?

BTW...Milgram’s experiment was a social psychology experiment. Social psychology can be sociology or psychology depending on how you look at it. This kind of experiment is no longer deemed ethical, which is why we only get observational or voluntary data now. I was taught that the kind of study that sociologists CAN do now will never be as “good”, data-wise, as they were back when they were, ahem, doing unethical things.


No it is not "easily" done. Easily done controls are usually implemented because if it is easy you do want that confidence.

If you're going to criticize Milgram's experiment, please give the proper control experiment to validate the results.


_________________
"Ignorance may be bliss, but knowledge is power."