How highly cited papers affect LIS journal impact factor? An empirical study

• The effect of highly cited papers of the LIS field on JIF and journal rankings to varying degrees was evaluated. • The Citation Concentration Index (CCI) serves as a diagnostic analytical tool for interpreting academic evaluation results derived from JIF. • An interpretation of journal rankings based on JIF should be combined with the citation distribution of papers within each journal. • The positive impact of highly cited papers on JIF and journal rankings cannot be ignored, particularly the strong effect of the proportion of highly cited papers from the previous year on the current year’s journal evaluation. The Journal Impact Factor (JIF) is a predominant indicator in academic evaluation. However, its validity may be compromised by the highly skewed distribution of citations within the same journal, which can lead to misinterpretations in research assessment. In light of these concerns, this study examines journals in the field of Library and Information Science (LIS) by applying different treatments of highly cited papers to evaluate their influence on JIF values and ranking outcomes. To enhance comprehension of JIF-based evaluation results, the Citation Concentration Index (CCI) is proposed, which reflects the distribution of citation counts among the papers in a journal. The analysis reveals that highly cited papers are more prevalent in top-ranked journals and exert a substantial influence on JIF, while for certain lower-ranked journals, the citation counts are all relatively low, and their JIFs are more dominated by a very small number of papers. These findings underscore that JIF-based journal rankings should be interpreted in conjunction with the citation distribution of papers within each journal.