The Impact of Open Science – what do we really know?

Abstract

Introduction
Open Science (OS) is increasingly mainstream, with wide adoption of practices like Open Access and FAIR data. While systems and data sources for monitoring the uptake of these practices are now relatively mature, research to establish real-world, long-term impacts is largely scattered and patchy.

In this talk, I (on behalf of the PathOS project team) will present work to scope and synthesise current evidence of:

1. Academic impact: E.g., OS contributions to research efficiency, quality, collaboration, and equity.
2. Societal impact: E.g., effects on policy, governance, education, climate, public trust, and health.
3. Economic impact: E.g., influence on industry productivity, innovation, and cost efficiency.

We used the PRISMA-SCR methodology (Tricco et al. 2018) to systematically scope literature from 2000-2022. Starting from a total of more than 30,000 initially retrieved records, the study team identified, selected, analysed, and synthesised peer-reviewed literature from academic databases and further “grey literature”/preprints from the Web.

Full results for two studies (academic and societal impact) are already available online (Cole et al. 2024; Klebel et al. 2024). The third, on economic impact, is in preparation and will soon be made available as a preprint and submitted for journal publication.

Results
415 studies demonstrated academic impact, 196 societal impact, and 70 economic impact. Most academic studies focus on Open Access, showing increased citation rates but raising equity concerns due to the article processing charge (APC) model. Citizen Science improves data collection and education but has varying levels of data quality. Open/FAIR data leads to reuse benefits and better reproducibility. Societal impact studies highlight Citizen Science’s role in education, climate awareness, and policy engagement, though Open Access’s societal impact remains limited. Economic studies suggest theoretical benefits like cost savings and innovation, but empirical data is scarce.

Discussion and conclusion
This talk will detail and reflect upon the these findings, including the uneven distribution of research on OS impacts, difficulties in assessing OS impact and establishing causality, the tendency for research to focus on areas where data is evenly available rather than where evidence is most needed, prior focus on uptake rather than impact, and the implications of all this for our understanding OS impact. Finally we will look ahead to PathOS’ future work on OS impact indicators, methods for cost-benefit analysis, and specific case-study investigations to deepen our understanding of OS’s influence on research efficiency, quality, equity, societal engagement, policy-making, and economic outcomes.