Percentile in Percentile: Evaluating Scholars Across Career Stages
arXiv:2103.16025
Abstract
Scholars are often evaluated early in their careers, before papers accumulate citations. Standard metrics such as citation counts and the -index fit these comparisons poorly: they grow with seniority, vary across fields and inflate over time. Although citation counts take years to build, a paper's citation percentile within its publication year settles quickly, correlating at 0.91 between ages 3 and 10. We introduce Percentile in Percentile (PiP), a score rating scholars against their career cohort. At each impact threshold above a cutoff, PiP counts a scholar's papers reaching that citation percentile, ranks that count against peers who started in the same year, and averages these standings. Across 11.1 million Semantic Scholar authors, PiP's distribution stays stable across 1960 to 2015 cohorts, whereas the average -index at career age 10 rose about 40%. In every setting and across all 15 fields, PiP outperforms the -index at predicting a scholar's record in the next decade, including future top papers, exceptional papers and overall citation impact, and reaches its age-10 accuracy in identifying future high performers 1.6 to 1.8 years earlier. Among all eligible authors, PiP also better anticipates later academy memberships, fellowships and prizes.
arXiv comment32 pages, 9 figures, 10 tables. Complete rewrite and expansion of earlier working paper, introducing the Percentile in Percentile (PiP) indicator with formal proofs and a full-scale reconstruction across 226M Semantic Scholar papers and 11.1M authors. Replication package: https://doi.org/10.5281/zenodo.23987119
References in corpus (11)
- Emergence of scaling in random networks
- An index to quantify an individual's scientific research output
- A Review of Theory and Practice in Scientometrics
- Does the h-index have predictive power?
- Quantifying Long-Term Scientific Impact
- Universality of citation distributions: towards an objective measure of scientific impact
- Finding Scientific Gems with Google
- Evolution of reference networks with aging
- Ranking Scientific Publications Using a Simple Model of Network Traffic
- Nonuniversal power law scaling in the probability distribution of scientific citations
- Modelling Aging Characteristics in Citation Networks