paper

Percentile in Percentile: Evaluating Scholars Across Career Stages

arXiv:2103.16025

Abstract

Scholars are often evaluated early in their careers, before papers accumulate citations. Standard metrics such as citation counts and the -index fit these comparisons poorly: they grow with seniority, vary across fields and inflate over time. Although citation counts take years to build, a paper's citation percentile within its publication year settles quickly, correlating at 0.91 between ages 3 and 10. We introduce Percentile in Percentile (PiP), a score rating scholars against their career cohort. At each impact threshold above a cutoff, PiP counts a scholar's papers reaching that citation percentile, ranks that count against peers who started in the same year, and averages these standings. Across 11.1 million Semantic Scholar authors, PiP's distribution stays stable across 1960 to 2015 cohorts, whereas the average -index at career age 10 rose about 40%. In every setting and across all 15 fields, PiP outperforms the -index at predicting a scholar's record in the next decade, including future top papers, exceptional papers and overall citation impact, and reaches its age-10 accuracy in identifying future high performers 1.6 to 1.8 years earlier. Among all eligible authors, PiP also better anticipates later academy memberships, fellowships and prizes.

arXiv comment32 pages, 9 figures, 10 tables. Complete rewrite and expansion of earlier working paper, introducing the Percentile in Percentile (PiP) indicator with formal proofs and a full-scale reconstruction across 226M Semantic Scholar papers and 11.1M authors. Replication package: https://doi.org/10.5281/zenodo.23987119

References in corpus (11)