Guest blog post by Henrik Nilsson.
Originally conceived as a means to compare university library holdings, the journal impact factor (IF) concept is a bibliometric measure that quantifies citations at the journal level, and is used for assessment and ranking.
However, IFs are often gratuitously commandeered for use in ranking individual research papers and even researchers according to perceived impact and importance. Surely, goes the rationale, this must be a time-efficient and data-informed way to obtain an objective ranking in the matter at hand. Over time, the IF concept has come to enjoy a level of decisiveness and mystique not permitted to any other research performance indicator.
“But are impact factors really a good measure of past scientific performance and future potential?” asks mycologist Henrik Nilsson of the University of Gothenburg, Sweden, the lead author of a study recently published in MycoKeys.
“Having served in one too many committees that handed out research funding to systematics and taxonomy on the primary, and sometimes exclusive, basis of impact factors, I felt that the time had come to find out. Simply taking IFs on faith as a performance indicator whose explanatory power is asserted by fiat alone seems deeply unsatisfactory and, frankly, not very scientific.”
Co-author Kessy Abarenkov of the University of Tartu Natural History Museum, Estonia continues, “We brought empirical data to bear on the matter in the form of two data streams for the period 2000-2021: the discovery of fungal species using DNA sequences and the description of new species of fungi.
“These data allowed us to assess whether IFs scale to mycological discovery potential – for instance, are new species of fungi primarily discovered in high-IF journals? Do journals with low IFs really add little, and journals without IFs nothing, to systematic mycology? Because that is what is assumed when candidates are ranked according to IFs.”
The study found no meaningful correlation between IFs and mycological discovery potential. On the contrary, for the last 10 years, the majority of new fungal species were discovered and described in journals with IFs well below the mycological median. Species discovered by molecular means were subsequently recovered in journals of increasingly higher IFs, suggesting that taxonomic results find broader, high-IF use in the mycological community and beyond in a way not usually considered when assessing the impact of taxonomic contributions.
To some extent, different groups of fungi were targeted in high-IF journals compared to low-IF journals, hinting that attempts at suppressing low-IF research are tantamount to advocating an intentionally incomplete view of the fungal kingdom.
“But what resonated the most with me,” co-author Christian Wurzbacher of the Technical University of Munich chips in, “was the large proportion of fungi that were discovered and described entirely outside the IF system. Various national and regional mycological societies and their outlets spring to mind.
“These journals are often dismissed or patronized when candidates and proposals are evaluated, but our study shows that they certainly punch far above their perceived weight. It feels good to lend voice to all the hard work that’s behind these journals and their studies but that is not accorded the clout that is should.”
The study submits that funding agencies and hiring committees that insist on upholding IFs as a central funding and recruitment criterion in systematic mycology should consider using indicators such as research quality, productivity, outreach activities, review services for scientific journals, and teaching ability directly rather than using publication in high-IF journals as a proxy for these indicators. Such an approach would clearly be much more time-consuming, a dilemma to which the study offers no other remedial measure than letting the evaluation step take the time that it needs.
Co-author Alice Retter of the Leibniz Institute for Freshwater Ecology, Germany concludes, “I would say that IFs in systematic mycology demand more explanation than they provide. The trust reposed in IFs in systematic mycology seems largely misplaced and, in fact, often directly counterproductive. I hope that the mycological community will continue to break ranks and colour outside the lines, because for us, the IF concept is a performance indicator of the kind that impoverishes.”
Original source
Nilsson RH, Jansson AT, Wurzbacher C, Anslan S, Belford P, Corcoll N, Dombrowski A, Ghobad-Nejhad M, Gustavsson M, Gómez-Martínez D, Kalsoom Khan F, Khomich M, Lennartsdotter C, Lund D, Van Der Merwe B, Mikryukov V, Peterson M, Porter TM, Põlme S, Retter A, Sanchez-Garcia M, Svantesson S, Svedberg P, Vu D, Ryberg M, Abarenkov K, Kristiansson E (2024) 20 years of bibliometric data illustrates a lack of concordance between journal impact factor and fungal species discovery in systematic mycology. MycoKeys 110: 273-285. https://doi.org/10.3897/mycokeys.110.136048
