Data Scaling as Progressive Coverage of a Predictive Contribution Spectrum

Zihui Song, Shihao Ji, Hongxi Li +2

We investigate the hypothesis that real-data scaling laws are governed by progressive coverage of a latent predictive contribution spectrum rather than by token-frequency tails alo…