Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra

Roman Worschech, Bernd Rosenow

Neural scaling laws describe how the performance of deep neural networks scales with key factors such as training data size, model complexity, and training time, often following po…