59 citations · 73 across the 5 of their papers we have counts for
5 papers
SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs
Lijun Yu, Yong Cheng, Zhiruo Wang +10
In this work, we introduce Semantic Pyramid AutoEncoder (SPAE) for enabling frozen LLMs to perform both understanding and generation tasks involving non-linguistic modalities such…
MuSLAM: Multitask, Multilingual Speech and Language Models
Yong Cheng, Yu Zhang, Melvin Johnson +2
We present MuSLAM, a multilingual sequence-to-sequence model pre-trained jointly on unlabeled speech, unlabeled text and supervised data spanning Automatic Speech Recognition…
Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation
Yong Cheng, Ankur Bapna, Orhan Firat +3
Multilingual neural machine translation models are trained to maximize the likelihood of a mix of examples drawn from multiple language pairs. The dominant inductive bias applied t…
Examining Scaling and Transfer of Language Model Architectures for Machine Translation
Biao Zhang, Behrooz Ghorbani, Ankur Bapna +4
Natural language understanding and generation models follow one of the two dominant architectural paradigms: language models (LMs) that process concatenated sequences in a single s…
mSLAM: Massively multilingual joint pre-training for speech and text
Ankur Bapna, Colin Cherry, Yu Zhang +6
We present mSLAM, a multilingual Speech and LAnguage Model that learns cross-lingual cross-modal representations of speech and text by pre-training jointly on large amounts of unla…