Making sense and making use of Altmetrics in research evaluation

Abstract

>> See video of presentation (35 min.)

 Monitoring and measuring the success of published research has long been a problematic area for institutions, funders and publishers. Recent government reviews, such as the Research Excellence Framework in the UK, have placed increasing demands on authors to provide substantial evidence of the academic and wider impact of their work. Over reliance on indices such as the Impact Factor has led to questionable tactics – with researchers seemingly choosing to submit work produced in the format that is most likely to have accrued impact demonstrated in the form of citations (i.e. published articles). But what about other outputs, and the other attention the work has received, and its application in real-world practices?

Altmetrics, or alternative metrics, have evolved in the last few years as an alternative to only using the Impact Factor, and offer a much richer picture of how a piece of research has been received. Instead of only looking at how the work is received within academia (most often via a citation count) altmetrics track the sharing and discussion of research in a much wider range of online sources – including global traditional and social media, post-publication peer-review forums, and public policy documents.

A focus on the qualitative data rather than the quantitative enables a better understanding of where and why the work is being discussed and potentially put into practice. It also offers a chance to focus on an individual article instead of making an assumption based on the perceived quality of the journal it is published in – a key pain point for those involved in research evaluation.

A relatively new field, there is still much work to be done into what, why, and how the attention surrounding a paper should (or should not) be measured. There is already a justified concern that they will simply become another number by which a researcher can be judged, and a danger that they will be used to indicate quality where in reality they can offer nothing of the sort.

What they do provide, however, is a valuable insight into the wider reception of research, and in a much more immediate way than citation counts alone are able to give. New tools are being developed at a rapid pace to enable institutions and funders to more easily track where the research their authors publish is being picked up – to help identify success stories and to further validate their knowledge of their audiences.

We will give a brief overview of the progress within the field of altmetrics to date, including tangible use cases amongst academics and institutional managers, and explore how by pulling together academic impact (the more traditional bibliometrics) and the wider societal impact (altmetrics) it is possible to benefit from delving deeper into the context and engagement surrounding research.