Neural Scaling Laws Rooted in the Data Distribution

Ari Brill

Deep neural networks exhibit empirical neural scaling laws, with error decreasing as a power law with increasing model or data size, across a wide variety of architectures, tasks,…