Interpretation and inference for altmetric indicators arising from sparse data statistics

Abstract

In 2018 Bornmann and Haunschild (2018a) introduced a new indicator called the Mantel-Haenszel quotient (MHq) to measure alternative metrics (or altmetrics) of scientometric data. In this article we review the Mantel-Haenszel statistics, point out two errors in the literature, and introduce a new indicator. First, we correct the interpretation of MHq and mention that it is still a meaningful indicator. Second, we correct the variance formula for MHq, which leads to narrower confidence intervals. A simulation study shows the superior performance of our variance estimator and confidence intervals. Since MHq does not match its original description in the literature, we propose a new indicator, the Mantel-Haenszel row risk ratio (MHRR), to meet that need. Interpretation and statistical inference for MHRR are discussed. For both MHRR and MHq, a value greater (less) than one means performance is better (worse) than in the reference set called the world.

arXiv commentTo appear in the Journal of Informetrics

References in corpus (6)

• Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data
• Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK Research Excellence Framework (REF)
• Three practical field normalised alternative indicator formulae for research evaluation
• Does the public discuss other topics on climate change than researchers? A comparison of explorative networks based on author keywords and hashtags
• Social media in scholarly communication
• Discrete power law with exponential cutoff and Lotka's Law