2 studies cast doubt on credibility of medical research

Two studies released Monday found major flaws with a large number of research papers in the biomedical sciences, a problem that authors say wastes billions and slows the pace of life-saving research.

One of the studies in the journal PLOS Biology analyzed a random sample of 441 biomedical journal articles published in the last 15 years, and found that only one included a full protocol of the information necessary for evaluating and replicating the research.

Scientific studies are considered valid only if they can be reproduced by outside teams of researchers who come to the same conclusions using independent methods.

Furthermore, most of the studies did not say where their funding came from, and did not establish whether there were any conflicts of interest.

"We hope our survey will further sensitize scientists, funders, journals and other science-related stakeholders about the need to improve these indicators," wrote the study authors, who included prominent scientist John Ioannidis from the Meta-Research Innovation Center at Stanford University and Shareen Iqbal from Emory University.

A separate study by researchers in Germany examined hundreds of published experiments on stroke and cancer research and found that most did not contain sufficient information about how many animals were used.

In many papers, the number of animals used over the course of the "vanished," making any conclusions far less reliable.

"The study began with an attempt to look at the robustness of findings in a handful of preclinical papers" said first author Constance Holman, a researcher at Charite Universitatsmedizin.

"But the sheer number of missing animals stopped us in our tracks."

When human subjects are part of a clinical trial, information about how many took part is considered crucial to the findings, and would not be left out of the published study, the authors said.

An accompanying editorial in PLOS Biology noted that a "credibility crisis" is ravaging scientific research.

For instance, a recent attempt to replicate 100 psychology studies found that only 39 percent could in fact be reproduced.

Other previous research has found that "85 percent of research investment in the biomedical sciences -– or US $200 billion of the worldwide investment in 2010 -– is wasted," it said.

The journal is launching a new Meta-Research Section aimed at improving standards.

"With our new section on data-driven meta-research, we aim to highlight that research about research is an important area of science," wrote editor Stavroula Kousta.

"By creating a prominent forum in this field, PLOS Biology will contribute to ongoing efforts to improve research standards in the biological sciences and beyond."