297 citations · 1.8k across the 50 of their papers we have counts for
6 papers · 2 filters
Joint Unsupervised and Supervised Training for Multilingual ASR
Junwen Bai, Bo Li, Yu Zhang +4
Self-supervised training has shown promising gains in pretraining models and facilitating the downstream finetuning for speech recognition, like multilingual ASR. Most existing met…
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training
Ankur Bapna, Yu-an Chung, Nan Wu +7
Unsupervised pre-training is now the predominant approach for both text and speech understanding. Self-attention models pre-trained on large amounts of unannotated data have been h…
Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference
Sneha Kudugunta, Yanping Huang, Ankur Bapna +4
Sparse Mixture-of-Experts (MoE) has been a successful approach for scaling multilingual translation models to billions of parameters without a proportional increase in training com…
Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents
Biao Zhang, Ankur Bapna, Melvin Johnson +3
Document-level neural machine translation (DocNMT) achieves coherent translations by incorporating cross-sentence context. However, for most language pairs there's a shortage of pa…
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Julia Kreutzer, Isaac Caswell, Lisa Wang +49
With the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, web-mined text dataset…
Gradient-guided Loss Masking for Neural Machine Translation
Xinyi Wang, Ankur Bapna, Melvin Johnson +1
To mitigate the negative effect of low quality training data on the performance of neural machine translation models, most existing strategies focus on filtering out harmful data b…