Practical Scaling Laws: Converting Compute into Performance in a Data-Constrained World

Christopher M. Bryant, Hao Liu

The scaling laws guiding modern model training were calibrated for a single regime: data-rich, single-epoch pretraining. The dominant such scaling law form, Chinchilla's $L = E + A…