5.7k citations · 7.1k across the 26 of their papers we have counts for
7 papers · 1 filter
DOCmT5: Document-Level Pretraining of Multilingual Language Models
Chia-Hsuan Lee, Aditya Siddhant, Viresh Ratnakar +1
In this paper, we introduce DOCmT5, a multilingual sequence-to-sequence language model pretrained with large scale parallel documents. While previous approaches have focused on lev…
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training
Ankur Bapna, Yu-an Chung, Nan Wu +7
Unsupervised pre-training is now the predominant approach for both text and speech understanding. Self-attention models pre-trained on large amounts of unannotated data have been h…
HintedBT: Augmenting Back-Translation with Quality and Transliteration Hints
Sahana Ramnath, Melvin Johnson, Abhirut Gupta +1
Back-translation (BT) of target monolingual corpora is a widely used data augmentation strategy for neural machine translation (NMT), especially for low-resource language pairs. To…
Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents
Biao Zhang, Ankur Bapna, Melvin Johnson +3
Document-level neural machine translation (DocNMT) achieves coherent translations by incorporating cross-sentence context. However, for most language pairs there's a shortage of pa…
XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation
Sebastian Ruder, Noah Constant, Jan Botha +8
Machine learning has brought striking advances in multilingual natural language processing capabilities over the past year. For example, the latest techniques have improved the sta…
They, Them, Theirs: Rewriting with Gender-Neutral English
Tony Sun, Kellie Webster, Apu Shah +2
Responsible development of technology involves applications being inclusive of the diverse set of users they hope to support. An important part of this is understanding the many wa…