297 citations · 1.8k across the 50 of their papers we have counts for
4 papers · 1 filter
Scaling Laws for Neural Machine Translation
Behrooz Ghorbani, Orhan Firat, Markus Freitag +5
We present an empirical study of scaling properties of encoder-decoder Transformer models used in neural machine translation (NMT). We show that cross-entropy loss as a function of…
Controlling Computation versus Quality for Neural Sequence Models
Ankur Bapna, Naveen Arivazhagan, Orhan Firat
Most neural networks utilize the same amount of compute for every example independent of the inherent complexity of the input. Further, methods that adapt the amount of computation…
Faster Transformer Decoding: N-gram Masked Self-Attention
Ciprian Chelba, Mia Chen, Ankur Bapna +1
Motivated by the fact that most of the information relevant to the prediction of target tokens is drawn from the source sentence , we propose truncating the tar…
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Jonathan Shen, Patrick Nguyen, Yonghui Wu +88
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models. Lingvo models a…