63.6k citations · 63.9k across the 10 of their papers we have counts for
11 papers
Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models
Luke Vilnis, Yury Zemlyanskiy, Patrick Murray +2
Decoding methods for large language models often trade-off between diversity of outputs and parallelism of computation. Methods such as beam search and Gumbel top-k sampling can gu…
Scaling Up Models and Data with and
Adam Roberts, Hyung Won Chung, Anselm Levskaya +40
Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves. Scaling can…
FRUIT: Faithfully Reflecting Updated Information in Text
Robert L. Logan, Alexandre Passos, Sameer Singh +1
Textual knowledge bases such as Wikipedia require considerable effort to keep up to date and consistent. While automated writing assistants could potentially ease this burden, the…
Faster Neural Network Training with Data Echoing
Dami Choi, Alexandre Passos, Christopher J. Shallue +1
In the twilight of Moore's law, GPUs and other specialized hardware accelerators have dramatically sped up neural network training. However, earlier stages of the training pipeline…
TensorFlow Eager: A Multi-Stage, Python-Embedded DSL for Machine Learning
Akshay Agrawal, Akshay Naresh Modi, Alexandre Passos +8
TensorFlow Eager is a multi-stage, Python-embedded domain-specific language for hardware-accelerated machine learning, suitable for both interactive research and production. Tensor…
Large scale distributed neural network training through online distillation
Rohan Anil, Gabriel Pereyra, Alexandre Passos +3
Techniques such as ensembling and distillation promise model quality improvements when paired with almost any base model. However, due to increased test-time cost (for ensembles) a…