Dynamics of neural scaling laws in random feature regression with powerlaw-distributed kernel eigenvalues

Jakob Kramp, Javed Lindner, Moritz Helias

Training large neural networks exposes neural scaling laws for the generalization error, which points to a universal behavior across network architectures of learning in high dimen…