On the Validity of Metacritic in Assessing Game Value

Abstract

The website Metacritic, which aggregates published reviews of various entertainment media into single “metascores,” has in recent years become highly influential in the area of video games, both with respect to consumers and industry members. This has resulted in an increase both in the attention paid to Metacritic and the level of criticism leveled at the institution. The present work examines the scientific validity of Metacritic as a measure of game quality and value, from both a qualitative and a quantitative perspective. Evidence for a strong correlational link between game sales and Metacritic metascores is presented, along with a detailed analysis of a number of threats to validity identified in the metascore production process. The present work concludes that while Metacritic suffers from a number of serious threats to validity, it remains an important tool for the industry if used correctly.