Do economics journals foster replicable research?




Watch the VIDEO here.

Replications are pivotal for the credibility of empirical economics. It is widely recognized in economics that replication studies are a necessary condition for scientific integrity. Alarmingly, several studies indicate that a significant share of empirical economics research cannot be replicated. At the same time, the awareness among researchers, that empirically-based research is often based on shaky grounds, has increased in the last years. It becomes more and more evident that there is a need for more replications in economics to regain trust and credibility in empirical economics research.

Though established scholarly journals have adopted replication policies in recent years, replication activities only slightly increased. Against this background our talk investigates if and how journals in economics foster replicable research. For this purpose, we will address two aspects:

  1. Journals’ data policies and their effective enforcement in economics: The first part of our talk presents the findings of a new study, in which we evaluated almost 600 articles published in 37 well-regarded journals with a data availability policy. First, we highlight the share of articles that fall under the data policy. Subsequently, the talk contrasts for how many of these data-based articles replication files were available in journals’ data archives and/or the supplemental information section of the article. Moreover, the exact requirements of journals’ data policies have been contrasted to the replication files available on journals’ web pages (respectively in their data archives).We developed a ‘compliance rate’ for each journal in our study. The higher the compliance rate the more do journals enforce their data policy. In the first part of our talk, we also discuss the question whether voluntary data policies are effective in fostering replicable research. For this purpose, we compare the compliance rate of journals with a voluntary data policy to their mandatory counterparts.
  2. Journals as publication outlets for replication studies: Though researchers agree that replication studies are needed to regain trust and credibility in empirical economic research, replication activities only slightly increased in recent years.  One reason for that finding can be that in the current system replicating other people’s result does not progresses researcher’s career. Another reason is the paucity of publication outlets for such replication studies. In this part of our talk we also discuss whether established journals should implement replication sections or whether a journal which is entirely dedicated to replication, would be a better way to foster the publication of replication studies. As a showcase we will briefly introduce the newly founded “International Journal for Re-Views in Empirical Economics” (IREE).

To conclude, we sketch the current and potential future developments in economics when it comes to reproducible research. 

Author Biographies

Sven Vlaeminck, Leibniz Information Centre for Economics (ZBW)

Sven is a project manager for reaserach data management at the Leibniz Information Centre for Economics (ZBW) in  Hamburg/Germany since 2010. He also acts as product manager of the ZBW Journal Data Archive, a newly established service for academic journals in economics and management.

Beyond, Sven is involved in da|ra – the German DOI-registration agency for social and economic data, which is part of the DataCite consortium.

He holds a degree in political sciences with a focus on quantitative social science research.

Ralf Toepfer, Leibniz Information Centre for Economics (ZBW)

Ralf Toepfer studied economics and socioeconomics at the Hamburg University for Economics and Politics in Hamburg, Germany. Since 2007 he is working at the ZBW Leibniz-Information-Centre for Economics, Kiel/Hamburg. He is the deputy head of the department "Publication Services" and he is involved in several research data related projects and services.

Sven Vlaeminck and Ralf Toepfer




How to Cite

Vlaeminck, S., & Toepfer, R. (2017). Do economics journals foster replicable research?. Septentrio Conference Series, (1).