Identification of milestone papers through time-balanced network centrality

Abstract

Citations between scientific papers and related bibliometric indices, such as the $h$-index for authors and the impact factor for journals, are being increasingly used - often in controversial ways - as quantitative tools for research evaluation. Yet, a fundamental research question remains still open: to which extent do quantitative metrics capture the significance of scientific works? We analyze the network of citations among the $449,935$ papers published by the American Physical Society (APS) journals between 1893 and 2009, and focus on the comparison of metrics built on the citation count with network-based metrics. We contrast five article-level metrics with respect to the rankings that they assign to a set of fundamental papers, called Milestone Letters, carefully selected by the APS editors for "making long-lived contributions to physics, either by announcing significant discoveries, or by initiating new areas of research". A new metric, which combines PageRank centrality with the explicit requirement that paper score is not biased by paper age, is the best-performing metric overall in identifying the Milestone Letters. The lack of time bias in the new metric makes it also possible to use it to compare papers of different age on the same scale. We find that network-based metrics identify the Milestone Letters better than metrics based on the citation count, which suggests that the structure of the citation network contains information that can be used to improve the ranking of scientific publications. The methods and results presented here are relevant for all evolving systems where network centrality metrics are applied, for example the World Wide Web and online social networks. An interactive Web platform where it is possible to view the ranking of the APS papers by rescaled PageRank is available at the address \url{http://www.sciencenow.info}.

arXiv commentMain text (pp. 1-12) + Appendices (pp. 13-18)

References in corpus (24)

• An index to quantify an individual's scientific research output
• Recommender Systems
• Quantifying Long-Term Scientific Impact
• Universality of citation distributions: towards an objective measure of scientific impact
• Finding Scientific Gems with Google
• Defining and identifying Sleeping Beauties in science
• Citation Statistics from 110 Years of Physical Review
• Journal Status
• Diffusion of scientific credits and the ranking of scientists
• Ranking Scientific Publications Using a Simple Model of Network Traffic
• The first-mover advantage in scientific publication
• Attention decay in science
• Google matrix analysis of directed networks
• Temporal effects in the growth of networks
• The Scientific Competitiveness of Nations
• Promise and Pitfalls of Extending Google's PageRank Algorithm to Citation Networks
• Rescaling citations of publications in physics
• A reverse engineering approach to the suppression of citation biases reveals universal properties of citation distributions
• Universality of scholarly impact metrics
• National Scientific Facilities and Their Science Impact on Non-Biomedical Research
• Prediction of highly cited papers
• Ranking nodes in growing networks: When PageRank fails
• Testing the fairness of citation indicators for comparison across scientific domains: the case of fractional citation counts
• Quality versus quantity in scientific impact

Cited by in corpus (10)

• Ranking in evolving complex networks
• Early identification of important patents through network centrality
• Quantifying and suppressing ranking bias in a large citation network
• Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data
• The essential role of time in network-based recommendation
• Randomizing growing networks with a time-respecting null model
• The long-term impact of ranking algorithms in growing networks
• Network-based ranking in social systems: three challenges
• Collective dynamics behind success
• Leveraging GANs for citation intent classification and its impact on citation network analysis