3k citations · 7.7k across the 20 of their papers we have counts for
Showing 2020Show all
3 papers · 1 filter
cs.LG2020★ 45 cited
Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images
Rewon Child
We present a hierarchical VAE that, for the first time, generates samples quickly while outperforming the PixelCNN in log-likelihood on all natural image benchmarks. We begin by ob…
cs.CL2020★ 3k cited
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typicall…
cs.LG2020★ 1.5k cited
Scaling Laws for Neural Language Models
Jared Kaplan, Sam McCandlish, Tom Henighan +7
We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute us…