Communicating uncertainty of scientific studies: focusing on 50 shades of gray rather than an accept-and-reject world

Abstract

Evidence from most studies is based on a black and white requirement of statistical significance, completely neglecting the uncertainty. This deterministic thinking is problematic because statistical significance on its own tells nothing about the magnitude of the effect, its practical significance, and the uncertainty around this evidence, thereby resulting in a high chance of making an interpretation mistake or taking the wrong decision. Through examples, we highlight the problems when assessing scientific evidence under a deterministic thinking and suggest how we can move from this black and white razor blade to a continuous gradient of 50 shades of grey, which allows for a more holistic assessment of evidence. We propose judging scientific evidence by focusing on the uncertainty around the evidence. We show how uncertainty can be presented and assessed with confidence intervals using an example from weather forecasting, by calculating the risks of Type S (sign) and Type M (magnitude) errors using an example from survival analysis, and by evaluating the reliability and validity of the results obtained using an example from wildlife management. We hope to raise awareness and prompt researchers to build better study designs, take better measurements, and make better use of statistics for inferring and judging scientific evidence.