Nordic Perspectives on Open Science 2021

Introducing level X in the Norwegian Publication Indicator: Involving the research community when evaluating journals operating in the borderland between predatory and reputable practice

Vidar Røeggen
Universities Norway

The Norwegian Publication Indicator

By introducing the Norwegian Publication Indicator in 2004 Norway became part of an international development in which the allocation of basic funds to research institutions is increasingly linked to performance indicators (Dansk center for forskningsanalyse, 2014). Denmark and Finland have also implemented what is frequently labeled as “The Norwegian Model”. The model has inspired changes in similar national models in Flanders (Belgium) and Poland, and it is used for local purposes by several universities in Sweden and by University of Dublin, in Ireland (Sivertsen, 2018). The research community has been deeply involved in designing and adopting the model in Norway, and the annual processes evaluating journals depend on involvement by panels in every field of research. The indicator has an interactive webpage where researchers can communicate and discuss publication channels openly, and the final decisions made by panels when nominating journals to the highest level (level 2) are transparent and openly available at the webpage.

The indicator depends on information from a national registry of approved publication channels that is managed by The Directorate of Higher Education and Skills (HK-dir.). As of November 2021, The Norwegian register for scientific journals, series and publishers contains 26 127 journals at the basic level (level 1) and 2 193 journals at the highest level (level 2), and level 2 journals are identified by research panels in 84 different fields of research. Researchers can suggest new publication channels to the registry and these suggestions are examined according to our four criteria:

Journals/series must:

  1. Be identified with a valid ISSN, confirmed by The International ISSN Register (demand from 2014)
  2. Have an academic editorial board (or an equivalent) primarily consisting of researchers from universities, institutes or organizations that do research
  3. Have established procedures for external peer review
  4. Have a national or international authorship, meaning that maximum 2/3 of the authors can belong to the same institution

Publishers must:
  1. Be organized in an editorial way to publish publications in accordance with the definition of a scientific publication
  2. Have a scientific publishing program with external advisors and aiming for distribution to scholars and research institutions
  3. Have a national or international authorship, meaning that maximum 2/3 of the authors can belong to the same institution

New suggestions are prepared by the secretariate at the register, and then finally approved by The National Board of Scholarly Publishing (NPU). So, the research community is deeply involved, both in the operations and further development of the indicator.

The secretariate at HK-dir. processes approximately 1 600 new proposals annually and NPU observe a new tendency in recent years: that an ever-increasing number of the incoming suggestions represents channels where there is uncertainty about approval or rejection. On the one hand, an examination of the available information on these journals’ webpages shows that the journals apparently satisfy our criteria. However, NPU sometimes identify ongoing discussion in the research community as to whether editorial practice is in accordance with how the journals describe their own routines. In addition, researchers often inform both NPU and the secretariate at HK-dir. about their own (bad) experience with a journal and ask us to investigate further.

Researchers often refer to these journals as “predatory journals” or the activity they represent as “predatory publishing”. But what does predatory publishing mean in 2021? The term has been co-opted to describe a range of activities including lack of rigorous peer review to exploitative publishing models (Hanson, 2021). Journals or publishers are not either predatory or representatives of high standards – they are rather on a continuum from predatory to high standards of research integrity and practice. Therefore, NPU discuss where to draw the line on this continuum.

Introducing Level X

A large amount of the journals that are up for discussions have been established based on the relatively new business model in open access publishing, charging fees from the authors (article processing charges). This model also has some policy implication when discussing research quality, because we can observe that installing payment on the same side as the ever-persistent pressure to publish makes publishers compete for volume rather than the quality of editorial procedures and contents.

To address this challenge, NPU decided to ask for input from our research community. We have established a list of journals categorized as level X, journals where there is uncertainty about approval or rejection. This list is published open at our webpage, and we invite researchers to participate with comments and inputs. We hope that comment, input, and open involvement from the research community will help us and provide information that can strengthen and legitimize our final decision.

The list of journals on the level X list is a mixture of journals. The first category are journals that previously have been approved, journals where Norwegian researchers have published their results. What is common for these journals is that we have received concerns, worries and stories of bad experience from researchers, either as authors, reviewers or when journals have contacted researchers inviting them into their editorial team, while the scope of the journals is far outside their core competence. The second category are journals that have not been approved yet – these are typically journals that meet our criteria, but where we have identified that a journal is already debated in different fora or blogs, or some index services have excluded the journal based on editorial misconduct.

The need to reinstall the term “recommendable journal”

Introducing level X has also actualized the need to remind ourselves of an overall principle that has been there since the register first was established. The register is not only a tool, but it should also serve quality standards in Norwegian research and cover journals and publishers that are recommendable from this perspective. NPU wants to make the basic principle for including journals and publishers more explicit with the term “recommendable journals”. The term indicates that academic values and concerns about research quality are behind the selection, not just formal criteria.

The panels have actively selected the journals at level 2 in their field. These are the journals they perceive as leading publication channels in a wide variety of academic contexts, publishing the most outstanding works by researchers from different countries in their field of research. Journals at level 1 are not actively selected by the panels in the same way. They are submitted as proposals to the registry and have been added to the list at level 1 if they meet our four technical criteria. Experience shows that it may be needed to involve the academic panels more systematically in this selection process. The panels will only judge whether they find the publication channel recommendable or not to publish in for their field of research. They will not be asked to give the reasons for their judgement.

By this change of procedures, we will take steps towards a registry that only covers journals that are recommendable from the point of view of national panels representing all Norwegian researchers in their field. The NPI system already allows for making all decisions transparent for the communities.


Dansk Center for Forskningsanalyse (2014). Evaluation of the Norwegian Publication Indicator. Available at:

Hanson, Mark A. (2021): My blog: so far, a mix of fruit flies and scientific publishing ethics:

Sivertsen, G (2018), The Norwegian Model in Norway, Journal of Data and Information Science, Vol. 3 No. 4, 2018, pp 3-19.