Neural Neural Scaling Laws

Michael Y. Hu, Jane Pan, Ayush Rajesh Jhaveri +2

Neural scaling laws predict how language model performance improves with increased training inputs. While aggregate metrics like validation loss can follow smooth power-law curves,…