Diverse Pretrained Context Encodings Improve Document Translation

Domenic Donato, Lei Yu, Chris Dyer

We propose a new architecture for adapting a sentence-level sequence-to-sequence transformer by incorporating multiple pretrained document context signals and assess the impact on…