OpenUP Measuring Research Impact: Concepts, Methods, Limitations and Solutions




Impact, Metrics


Watch the VIDEO.

The immediate Return on Investment (ROI) for basic scientific research is scientific impact – improvements in knowledge of our physical, biological and social world. The Return on Investment (ROI) for applied research and the long-term Return on Investment for basic research is societal impact (i.e. impact on health, environment, the economy, etc.). However, many of these impacts are hard to measure and may only be apparent decades after the original investment. This creates demand from funders and policy makers for metrics that predict impacts before they can be measured. Here we define a preliminary conceptual framework describing the chain of events leading from the outputs of basic research  (publications, data, software, cell lines, equipment, methodologies, theories etc.) to the outputs of applied research (products, treatments, technology components etc.) to societal, financial, health and environmental impact. We go on to discuss how these impacts are currently measured in the short term (days and weeks), the medium term (years) and the long term (decades), and to identify the main providers of impact metrics. We highlight the different ways in which research metrics are used by different categories of user (researchers, institutions, national and European policymakers). Finally, we discuss the limitations of current metrics and possible solutions.

Author Biographies

Mappet Walker, Frontiers Media SA

A Programme, Project Manager, and Software Engineer, by discipline, Mappet is responsible for Planning and Projects at Frontier Media in Switzerland. For H2020 OpenUP project, she leads discussions on Research Impact and for OpenMinTED, she designed the text and data mining use cases in Archaeology, Research Impact, and Health State Modelling.

Stephanie Oeben, Frontiers Media SA

From a background in Japanese and Chinese Studies, Stephanie finished her Ph.D. in Anthropology about online video game communities at Oxford Brookes University in 2013. Until 2017, she worked for Elsevier on Open Access and Market Intelligence, and as an editor for Research Trends. She then joined Frontiers where she uses her expertise for data-driven decision making. 

Richard Walker, Frontiers Media SA

Richard has recently retired from his position as a staff scientist in EPFL's Blue Brain Project – whose goal is to build biologically detailed simulations of the rat and ultimately the human brain. Over a long career in industry and academia, Richard has published on a broad range of topics, including the role of peer review in modern scientific publishing. Since his retirement he has taken on a part time position as a senior consultant for Frontiers Media.