I take studies with a grain of salt.

Post Reply New Topic

I'm a logic-oriented person who is skeptical of things until I find evidence. That being said, the amount of times I've seen studies being treated like dogma just because they're studies is maddening. There are so many conditions needed in order to make a good study, and therefore, the majority of studies are strongly lacking in at least one area.

There are several questions I like to ask before I accept a study as truth.

* Is it conducted and endorsed by a university?
* Who is funding the study? Is it a biased organization?
* Is the sample size large enough to represent target demographic?
* How long did the study go on? Years? Months? An hour?
* Has the result in this study been replicated in other studies?
* Does the study prove causation, or only correlation?
* Are the metrics objective, or is the data gathered by self-reporting?
* Does the data actually prove the thesis the study wants to make? (Aka, "Boys got angrier while playing volleyball than girls did" therefore "Boys are always more aggressive than girls")

I do as well, think about how much science is proven completely wrong 10 or 20 years later and it's hard to understand how anybody can put blind faith into. It is a religion to some people.

I believe strongly in anecdote acting in complement with research studies.

Much of it certainly has become "mantra" by now.
Even Peer-Review is not exactly reliable either.
I generally prefer Independent-Research over Corporate-Funded.

You will want to use reviews and meta-analyses, not studies. If you want to find the reviews and meta-analysis on medical studies, you can go to pubmed.gov and go to advanced search and make sure to add type and select "Review" and "Meta-Analysis". Reviews and meta-analyses, as secondary sources, provide a much better overview of the research than single studies (or primary sources).

WikiProject Medicine requires their articles to be based on secondary sources.

A sensible position OP.
I keep myself to a similar position, and indeed, studies are given too much weight at times, by the media

These questions SHOULD be answered by peer review.

Unfortunately, we have a lot of problems with our current system of peer review.

As an example, studies that fail to prove a hypothesis rarely get written up or published and studies that check results of other studies struggle to get funding or have results published...

As a result, certain subjects are focused on by only a few researchers. This can lead to unconscious bias and just generally sloppy research.

For example, last year, I did a research proposal for housing first programs for the chronically homeless. When I did my lit review, I found a ton of research, BUT it was all done by the same 4 or 5 people working together.
These people also ran the programs.
That's a problem.

I mean, these guys should be writing up and assessing their results, but other people should be looking at their program too.

However, getting a grant to keep these guys 'honest' would be extremely difficult.

Some people have suggested that journals should agree to publish study reports BEFORE the studies are made. This would, at least, get more negative findings published. That would be a good start.

Also, funding parallel studies (two teams researching the same thing at the same time, but independently) would help too.

Plus, it would give more researchers a job... That's good too.