The replication and reproducibility crises: origins and consequences for studies of ecology and evolution
Keywords:Reproducibility, P-value, uncertainty, novelty, hypothesis testing
Watch the VIDEO.
There is a widespread discussion around a scientific crisis, resulting from a lack of reproducibility of published scientific studies. This was exemplified by Ioannidis’ 2005 paper “Why most published research findings are false” or the 2015 Open Science Collaboration study assessing reproducibility of psychological science. An often-cited reason for this reproducibility crisis is a fundamental misunderstanding of what statistical methods, and in particular P-values, can achieve. In the context of studies of ecology and evolution, I will show how 1) the pressure for publishing “novel” results, 2) what Gelman has called the “garden of forking paths”, i.e. the fact that published analyses represent only one out of many possible analyses, and 3) the often fruitless dichotomy between a null and alternative hypotheses, has led to the present situation. While scientific progress is dependent of major breakthroughs, we also need to find a better balance between confirmatory research – understanding how known effects vary in size according to the context – and exploratory, non-incremental research – finding new effects.
Ioannidis, J. P. A. 2005. Why most published research findings are false. Plos Medicine 2:696-701. https://doi.org/10.1371/journal.pmed.0020124
Open_Science_Collaboration. 2015. Estimating the reproducibility of psychological science. Science 349.
Yoccoz, N. G. 1991. Use, overuse and misuse of significance tests in evolutionary biology and ecology. Bulletin of the Ecological Society of America 72:106-111.
How to Cite
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).