Publication metrics: Improved accuracy is necessary and attainable

Abstract

The primary purpose of metrics is to measure publication performance. Individual, institutional and national players involved in knowledge production all want to know how well they are doing in relation to others. Their shared concern is to avoid being misguided by flawed metrics. Nevertheless, a common flaw is that contemporary metrics ignore differential coauthor contribution, by allocating equal publication credit to all coauthors irrespective of their actual contribution. The ensuing equalizing bias (EqB) skews bibliometric assessments, and accounts for a massive shift of credit from primary to secondary authors.

Biased equal credit scores produce distorted publication performance rankings that are fundamentally different from rankings obtained from estimates of actual coauthor credit (Hagen 2014a). Furthermore, the powerful distortional effect of EqB is inevitably compounded in derived bibliometric indices and indicators. EqB may also provide an incentive for unethical behaviour, including unwarranted claims for “honorary authorship” or “institutional plagiarism”. Nonetheless, EqB is still universally unaccounted for in modern metrics-based research evaluation.

The key to more reliable publication metrics is accurate accreditation of coauthors, based on all relevant byline information in bibliometric source data. This task is facilitated by the harmonic formula, which provides a transparent and verifiable quantification of the byline hierarchy. The harmonic formula provides accuracy and parsimony by adhering to three simple ethical criteria for the equitable distribution of authorship credit, and also accommodates additional byline information indicating the equality of some or all coauthors, or the presence of a senior author.

A recent study found that the harmonic formula explained nearly 97% of the variation in a composite empirical dataset (Hagen 2013). In contrast, biased equal credit explained less than 40% of the variation, while EqB accounted for the approximately 60% discrepancy between the equal and harmonic estimates. A follow-up theoretical analysis showed that the amount of EqB increases as the number of coauthors increases, and suggested that most conventional credit scores are likely to be dominated by EqB when the number of coauthors exceeds 12 (Hagen 2014b). The implication is that a quantum leap in the accuracy of publication metrics is readily attainable, with minor source-level modification of existing information retrieval technology.

References (Open access)

Hagen, N.T. 2013. Harmonic coauthor credit: A parsimonious quantification of the byline hierarchy. Journal of Informetrics 7(4):784–791. http://dx.doi.org/10.1016/j.joi.2013.06.005

Hagen, N.T. 2014a. Counting and comparing publication output with and without equalizing and inflationary bias. Journal of Informetrics 8(2):310-317. http://dx.doi.org/10.1016/j.joi.2014.01.003

Hagen, N.T. 2014b. Reversing the byline hierarchy: The effect of equalizing bias on the accreditation of primary, secondary and senior authors. Journal of Informetrics 8(3):618-627. http://dx.doi.org/10.1016/j.joi.2014.05.003