The Accuracy of Confidence Intervals for Field Normalised Indicators
arXiv:1703.04031 · doi:10.1016/j.joi.2017.03.004
Abstract
When comparing the average citation impact of research groups, universities and countries, field normalisation reduces the influence of discipline and time. Confidence intervals for these indicators can help with attempts to infer whether differences between sets of publications are due to chance factors. Although both bootstrapping and formulae have been proposed for these, their accuracy is unknown. In response, this article uses simulated data to systematically compare the accuracy of confidence limits in the simplest possible case, a single field and year. The results suggest that the MNLCS (Mean Normalised Log-transformed Citation Score) confidence interval formula is conservative for large groups but almost always safe, whereas bootstrap MNLCS confidence intervals tend to be accurate but can be unsafe for smaller world or group sample sizes. In contrast, bootstrap MNCS (Mean Normalised Citation Score) confidence intervals can be very unsafe, although their accuracy increases with sample sizes.
arXiv commentJournal of Informetrics, in press
References in corpus (10)
- Power-law distributions in empirical data
- Regression for citation data: An evaluation of different methods
- Predicting the long-term citation impact of recent publications
- A reverse engineering approach to the suppression of citation biases reveals universal properties of citation distributions
- Three practical field normalised alternative indicator formulae for research evaluation
- Distributions for cited articles from individual subjects and years
- National research impact indicators from Mendeley readers
- Are there too many uncited articles? Zero inflated variants of the discretised lognormal and hooked power law distributions
- Citation count distributions for large monodisciplinary journals
- Conceptual difficulties in the use of statistical inference in citation analysis
Cited by in corpus (4)
- Coverage of highly-cited documents in Google Scholar, Web of Science, and Scopus: a multidisciplinary comparison
- Is Research Funding Always Beneficial? A Cross-Disciplinary Analysis of UK Research 2014-20
- What increases (social) media attention: Research impact, author prominence or title attractiveness?
- Confidence intervals for normalised citation counts: Can they delimit underlying research capability?