Model-based evaluation of scientific impact indicators
arXiv:1606.04319 · doi:10.1103/PhysRevE.94.032312
Abstract
Using bibliometric data artificially generated through a model of citation dynamics calibrated on empirical data, we compare several indicators for the scientific impact of individual researchers. The use of such a controlled setup has the advantage of avoiding the biases present in real databases, and allows us to assess which aspects of the model dynamics and which traits of individual researchers a particular indicator actually reflects. We find that the simple citation average performs well in capturing the intrinsic scientific ability of researchers, whatever the length of their career. On the other hand, when productivity complements ability in the evaluation process, the notorious and indices reveal their potential, yet their normalized variants do not always yield a fair comparison between researchers at different career stages. Notably, the use of logarithmic units for citation counts allows us to build simple indicators with performance equal to that of and . Our analysis may provide useful hints for a proper use of bibliometric indicators. Additionally, our framework can be extended by including other aspects of the scientific production process and citation dynamics, with the potential to become a standard tool for the assessment of impact metrics.
arXiv comment12 pages, 8 figures
References in corpus (21)
- Statistical mechanics of complex networks
- An index to quantify an individual's scientific research output
- A Review of Theory and Practice in Scientometrics
- Quantifying Long-Term Scientific Impact
- Universality of citation distributions: towards an objective measure of scientific impact
- An index to quantify an individual's scientific research valid across disciplines
- Citation Statistics from 110 Years of Physical Review
- Diffusion of scientific credits and the ranking of scientists
- Reputation and Impact in Academic Careers
- The Simultaneous Evolution of Author and Paper Networks
- Ranking Scientific Publications Using a Simple Model of Network Traffic
- A review of the characteristics of 108 author-level bibliometric indicators
- Characterizing and modeling citation dynamics
- The first-mover advantage in scientific publication
- Temporal effects in the growth of networks
- The distorted mirror of Wikipedia: a quantitative analysis of Wikipedia coverage of academics
- On the calculation of percentile-based bibliometric indicators
- Universality of scholarly impact metrics
- Exploiting citation networks for large-scale author name disambiguation
- Statistical validation of high-dimensional models of growing networks
- The effect of the initial network configuration on preferential attachment
Cited by in corpus (10)
- Ranking in evolving complex networks
- The coverage of Microsoft Academic: Analyzing the publication output of a university
- Quantifying and suppressing ranking bias in a large citation network
- Science and Facebook: the same popularity law!
- Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data
- Geography of Science: Competitiveness and Inequality
- Randomizing growing networks with a time-respecting null model
- The long-term impact of ranking algorithms in growing networks
- The Evolution of Competitiveness across Economic, Innovation and Knowledge production activities
- Viewing citation trend of Indian physics and astronomy research papers since 2005 through the lens of some new indicators