5.7k citations · 6.7k across the 20 of their papers we have counts for
22 papers
Reinforced Self-Training (ReST) for Language Modeling
Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan +11
Reinforcement learning from human feedback (RLHF) can improve the quality of large language model's (LLM) outputs by aligning them with human preferences. We propose a simple algor…
SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs
Lijun Yu, Yong Cheng, Zhiruo Wang +10
In this work, we introduce Semantic Pyramid AutoEncoder (SPAE) for enabling frozen LLMs to perform both understanding and generation tasks involving non-linguistic modalities such…
MuSLAM: Multitask, Multilingual Speech and Language Models
Yong Cheng, Yu Zhang, Melvin Johnson +2
We present MuSLAM, a multilingual sequence-to-sequence model pre-trained jointly on unlabeled speech, unlabeled text and supervised data spanning Automatic Speech Recognition…
Building Machine Translation Systems for the Next Thousand Languages
Ankur Bapna, Isaac Caswell, Julia Kreutzer +21
In this paper we share findings from our effort to build practical machine translation (MT) systems capable of translating across over one thousand languages. We describe results i…
Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation
Yong Cheng, Ankur Bapna, Orhan Firat +3
Multilingual neural machine translation models are trained to maximize the likelihood of a mix of examples drawn from multiple language pairs. The dominant inductive bias applied t…
Self-supervised and Supervised Joint Training for Resource-rich Machine Translation
Yong Cheng, Wei Wang, Lu Jiang +1
Self-supervised pre-training of text representations has been successfully applied to low-resource Neural Machine Translation (NMT). However, it usually fails to achieve notable ga…