Improved Scaling Laws in Linear Regression via Data Reuse

Licong Lin, Jingfeng Wu, Peter L. Bartlett

Neural scaling laws suggest that the test error of large language models trained online decreases polynomially as the model size and data size increase. However, such scaling can b…