Testing the fairness of citation indicators for comparison across scientific domains: the case of fractional citation counts

Abstract

Citation numbers are extensively used for assessing the quality of scientific research. The use of raw citation counts is generally misleading, especially when applied to cross-disciplinary comparisons, since the average number of citations received is strongly dependent on the scientific discipline of reference of the paper. Measuring and eliminating biases in citation patterns is crucial for a fair use of citation numbers. Several numerical indicators have been introduced with this aim, but so far a specific statistical test for estimating the fairness of these numerical indicators has not been developed. Here we present a statistical method aimed at estimating the effectiveness of numerical indicators in the suppression of citation biases. The method is simple to implement and can be easily generalized to various scenarios. As a practical example we test, in a controlled case, the fairness of fractional citation count, which has been recently proposed as a tool for cross-discipline comparison. We show that this indicator is not able to remove biases in citation patterns and performs much worse than the rescaling of citation counts with average values.

8 pages, 6 figures, 1 table

References in corpus (5)

• An index to quantify an individual's scientific research output
• Universality of citation distributions: towards an objective measure of scientific impact
• Citation Statistics
• Rescaling citations of publications in physics
• National Scientific Facilities and Their Science Impact on Non-Biomedical Research

Cited by in corpus (11)

• A Review of Theory and Practice in Scientometrics
• The Chaperone Effect in Scientific Publishing
• A reverse engineering approach to the suppression of citation biases reveals universal properties of citation distributions
• Universality of scholarly impact metrics
• Identification of milestone papers through time-balanced network centrality
• Quantifying and suppressing ranking bias in a large citation network
• Citation score normalized by cited references (CSNCR): The introduction of a new citation impact indicator
• In science "there is no bad publicity": Papers criticized in comments have high scientific impact
• Bibliometric Indicators of Young Authors in Astrophysics: Can Later Stars be Predicted?
• Completing $h$
• The Z-index: A geometric representation of productivity and impact which accounts for information in the entire rank-citation profile