Communicating uncertainty of scientific studies: focusing on 50 shades of gray rather than an accept-and-reject world
Evidence from most studies is based on a black and white requirement of statistical significance, completely neglecting the uncertainty. This deterministic thinking is problematic because statistical significance on its own tells nothing about the magnitude of the effect, its practical significance, and the uncertainty around this evidence, thereby resulting in a high chance of making an interpretation mistake or taking the wrong decision. Through examples, we highlight the problems when assessing scientific evidence under a deterministic thinking and suggest how we can move from this black and white razor blade to a continuous gradient of 50 shades of grey, which allows for a more holistic assessment of evidence. We propose judging scientific evidence by focusing on the uncertainty around the evidence. We show how uncertainty can be presented and assessed with confidence intervals using an example from weather forecasting, by calculating the risks of Type S (sign) and Type M (magnitude) errors using an example from survival analysis, and by evaluating the reliability and validity of the results obtained using an example from wildlife management. We hope to raise awareness and prompt researchers to build better study designs, take better measurements, and make better use of statistics for inferring and judging scientific evidence.
Copyright (c) 2018 Sandra Hamel, Nigel Gilles Yoccoz
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).