Nature's news article published a few days ago shows that the debate on the reproducibility of scientific experiments has reached a new level. Apparently, the issue got the attention of the science leadership at the National Institutes of Health (NIH), and they are currently discussing potential solutions. This is very important as the NIH is the main funder of academic biomedical research in US, distributing nearly $31 billion per year in research grants.
I previously wrote about the hair-rising figures on the reproducibility of scientific experiments. It appears that only 10% to 30% of published scientific studies are actually reproducible. Epithets like “crazy”, “unbelievable” and “impossible” pop up in the mind of every scientist when they read about these numbers, because science is not science without reproducibility.
The two main projects that determined these crazy numbers were conducted by the pharma/biotech companies Amgen (California) and Bayer (Germany). More than a year ago,their studies were published in Nature, and further publicized in the Wall Street Journal and other general media. Back then, contrary to my expectations, the skies did not immediately fall on the earth. The reaction in the academic community was quite mute, there was no outcry among the science-related organizations, and business went as usual. It seems that everyone pretended that nothing serious happened and the science debate went back to its familiar venues of “not enough funding for science,” “we don’t have enough PhDs,” “open access vs. subscriptions” and so on, and nobody actually cares about the new evidence that put into question the entire foundation on which the science and scientists exist. Some articles cutely called it science’s “dirty little secret.”
However, it seems that the reproducibility problem was not missed by the NIH leaders and they are now gathering information to determine the next steps. The Nature article brings a few serious names including Francis Collins, the current Director of NIH and one of the leading people behind the human genome sequencing, and Harold Varmus, the former NIH director, a Nobel Prize winner, who is also a co-founder of the successful open access publication PLoS. The potential solutions mentioned include random audits of published studies. This would be interesting! An introduction of such new methods of quality control may create significant changes into the practice of science and scientific publication. It remains unclear who will pay for such audits, for which the cost is estimated to reach $25,000 per published study. Given nearly 1,000,000 biomedical articles published every year, you do the math…
