Scaling Laws in Linear Regression: Compute, Parameters, and Data

Licong Lin, Jingfeng Wu, Sham M. Kakade +2

Empirically, large-scale deep learning models often satisfy a neural scaling law: the test error of the trained model improves polynomially as the model size and data size grow. Ho…