DOCmT5: Document-Level Pretraining of Multilingual Language Models

Chia-Hsuan Lee, Aditya Siddhant, Viresh Ratnakar +1

In this paper, we introduce DOCmT5, a multilingual sequence-to-sequence language model pretrained with large scale parallel documents. While previous approaches have focused on lev…