1.5k citations · 1.7k across the 3 of their papers we have counts for
3 papers
Scaling Laws for Neural Language Models
Jared Kaplan, Sam McCandlish, Tom Henighan +7
We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute us…
An Empirical Model of Large-Batch Training
Sam McCandlish, Jared Kaplan, Dario Amodei +1
In an increasing number of domains it has been demonstrated that deep learning models can be trained using relatively large batch sizes without sacrificing data efficiency. However…
Learning a Natural Language Interface with Neural Programmer
Arvind Neelakantan, Quoc V. Le, Martin Abadi +2
Learning a natural language interface for database tables is a challenging task that involves deep language understanding and multi-step reasoning. The task is often approached by…