Studies show very low reproducibility for articles published in scientific journals, often as low as 10-30%. Here is a partial list:
- The biotech company Amgen had a team of about 100 scientists trying to reproduce the findings of 53 "landmark" articles in cancer research published by reputable labs in top journals. Only 6 of the 53 studies were reproduced (about 10%).
- Scientists at the pharmaceutical company, Bayer, examined 67 target-validation projects in oncology, women's health, and cardiovascular medicine. Published results were reproduced in only 14 out of 67 projects (about 21%).
- The project, PsychFileDrawer, dedicated to replication of published articles in experimental psychology, shows a replication rate 3 out of 9 (33%) so far.
My hair is standing on end as I read these numbers! Unbelievable! The reproducibility of published experiments is the foundation of science. No reproducibility – no science. If these numbers are true, or even half-true, it means there is something fundamentally wrong in today's system of scientific research and education.
On a practical level, the US government gives nearly $31 billion every year in science funding through NIH only, which is mainly distributed in research grants to academic scientists. The 10% reproducibility rate means that 90% of this money ($28 billion) is wasted. That's a lot. How are the tax-payers supposed to respond to the scientist plight for more research funding given these numbers? Would you give more of your own money to someone who delivered you such a result?
Beyond the practicalities, there is an interesting philosophical question. Since the middle of the 20th century, life science research concepts and technologies have rapidly grown from the discovery of DNA to sequencing of genomes. Amazing technologies like microarrays, mass spectrometry, high-throughput assays, imaging, and robotic surgeries were introduced, making biology a data-rich science. One would expect that all these new tools would make science more rigorous and precise, but something opposite is happening.
Any ideas what could be the reason? Who is the main culprit? Write me directly at moshe.pritsker@jove.com.