It is how we know scientific findings are valid: if an experiment produces a result once, then it will likely produce it a second time, and a third, and so on – each time the experiment is repeated. However, some research studies produce results that cannot be reproduced.
Making research procedures more robust will underpin public trust in biomedical science, as well as leading to improved products, services – and health – for everyone. This is why 46 IAP for Health member academies today committed to action that will address issues of reproducibility: Their new statement, "A Call to Action to Improve the Reproducibility of Biomedical Research", outlines the causes of irreproducible results and recognises the leading role the academies can play in addressing this challenge in their own countries and worldwide.
“A Call to Action to Improve the Reproducibility of Biomedical Research” was released during the first day of IAP for Health's two-day conference in Beijing, "Promoting Health".
Results may be irreproducible simply because of natural variability in the biological systems being studied. Alternatively, they might be caused when scientists do not accurately report the details of how their experiment was carried out for others to follow, because the experimental designs were flawed, or even because the data produced were not analysed robustly enough.
Whatever the reasons, this lack of reproducibility in research is creating growing unease among scientists and the public, who are calling for more to be done to ensure that the quality of evidence is as high as it should be – including evidence from biomedical research.
"Now that these problems are widely recognised, we have an opportunity to unite and address the key areas highlighted in the IAP statement. The motivation is there for a concerted effort..."
Evidence of the scale of the reproducibility challenge is compelling. For example, the "Reproducibility Project: Psychology", which published its results last year, set out to reproduce 100 psychological studies and found that just 36% of its replication attempts produced meaningful results, while 97% of the original studies claimed to have done so. Other similar findings have emerged from fields such as cancer biology and drug development.
The academies highlight that research that cannot be reproduced wastes resources and delays the translation of results into clinical applications. It also has the potential to undermine the public’s trust in research findings, which threatens the reputation of biomedical science.
Lai Meng Looi, of the Academy of Sciences Malaysia and co-chair of IAP for Health, says: “The issue of irreproducible results affects researchers in all countries – high-income countries and low- and middle-income countries. By releasing this statement, we hope that IAP for Health and its member academies, representing the scientific community, will be able to raise awareness of the issue. We expect that our member academies will now present it directly to their national governments and disseminate it through their national scientific communities so that our recommendations can begin to be implemented.”
IAP for Health’s other co-chair, Detlev Ganten of the German National Academy of Sciences, Leopoldina, added: “Like all IAP statements, this statement on reproducibility has been thoroughly reviewed. It presents – based on the latest evidence from our member academies – the best impartial advice to policy-makers, whether at national or international level, as well as to healthcare and biomedical researchers, scientific publishers and funding agencies”
Among its recommendations, the statement urges a shift in the culture of science to better reward robust scientific methods and, when it comes to career advancement, to value the development of good research practice as much as new findings.
The statement recommends that funders set the tone for reproducible research by, for example, rigorously assessing proposed experimental designs, and it urges publishers and journal editors to enable greater openness and transparency by publishing replications of previous research or negative results. The Statement also encourages academies and other members of the scientific community to take steps to ensure that peer-review practices focus on the quality of the science more than on any potential excitement generated by the results.
Finally, the IAP for Health Statement encourages scientists to participate openly in the global dialogue around how to improve the reproducibility of results throughout the international research environment.
"Irreproducible results are often preventable and avoidable," adds Dorothy Bishop, professor of developmental neuropsychology at Oxford University, UK, and chair of the IAP for Health working group that prepared the statement. "Now that these problems are widely recognised, we have an opportunity to unite and address the key areas highlighted in the IAP statement. The motivation is there for a concerted effort from academies, the scientific community, funders, institutions and publishers across the world to strengthen the robustness of peer-reviewed medical research and maintain confidence in its findings."
The IAP for Health conference is hosted by the Chinese Academy of Engineering. The events are being organized in collaboration with the Chinese Academy of Medical Sciences/Peking Union Medical College.
About the InterAcademy Partnership for Health
IAP for Health is a component network of the InterAcademy Partnership. Its membership comprises 78 academies of medicine or academies of science and engineering with medical sections. It is committed to improving health worldwide, including through the release of consensus statements on matters of importance to global health. IAP for Health statements are prepared by a working group comprising experts nominated by member academies, and are released once they have been endorsed by more than half the member academies of the network.