3k citations · 11.9k across the 60 of their papers we have counts for
58 papers
Scaling Laws for Transfer
Danny Hernandez, Jared Kaplan, Tom Henighan +1
We study empirical scaling laws for transfer learning between distributions in an unsupervised, fine-tuning setting. When we train increasingly large neural networks from-scratch o…
Explaining Neural Scaling Laws
Yasaman Bahri, Ethan Dyer, Jared Kaplan +2
The population loss of trained deep neural networks often follows precise power-law scaling relations with either the size of the training dataset or the number of parameters in th…
Scaling Laws for Autoregressive Generative Modeling
Tom Henighan, Jared Kaplan, Mor Katz +16
We identify empirical scaling laws for the cross-entropy loss in four domains: generative image modeling, video modeling, multimodal imagetext models, and mathemat…
Causality Constraints in Large QCD Coupled to Gravity
Jared Kaplan, Sandipan Kundu
Confining gauge theories contain glueballs and mesons with arbitrary spin, and these particles become metastable at large . However, metastable higher spin particles, when coupl…
Closed Strings and Weak Gravity from Higher-Spin Causality
Jared Kaplan, Sandipan Kundu
We combine old and new quantum field theoretic arguments to show that any theory of stable or metastable higher spin particles can be coupled to gravity only when the gravity secto…
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typicall…