A Survey on Document-level Neural Machine Translation: Methods and Evaluation

Abstract

Machine translation (MT) is an important task in natural language processing (NLP) as it automates the translation process and reduces the reliance on human translators. With the resurgence of neural networks, the translation quality surpasses that of the translations obtained using statistical techniques for most language-pairs. Up until a few years ago, almost all of the neural translation models translated sentences independently, without incorporating the wider document-context and inter-dependencies among the sentences. The aim of this survey paper is to highlight the major works that have been undertaken in the space of document-level machine translation after the neural revolution, so that researchers can recognise the current state and future directions of this field. We provide an organisation of the literature based on novelties in modelling and architectures as well as training and decoding strategies. In addition, we cover evaluation strategies that have been introduced to account for the improvements in document MT, including automatic metrics and discourse-targeted test sets. We conclude by presenting possible avenues for future exploration in this research field.

arXiv commentAccepted for publication by ACM Computing Surveys

References in corpus (22)

• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Convolutional Sequence to Sequence Learning
• BERTScore: Evaluating Text Generation with BERT
• Sequence Transduction with Recurrent Neural Networks
• Multilingual Denoising Pre-training for Neural Machine Translation
• MASS: Masked Sequence to Sequence Pre-training for Language Generation
• From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
• Universal Transformers
• Bayes-Ball: The Rational Pastime (for Determining Irrelevance and Requisite Information in Belief Networks and Influence Diagrams)
• Does Neural Machine Translation Benefit from Larger Context?
• Modeling Coherence for Neural Machine Translation with Dynamic and Topic Caches
• Translationese in Machine Translation Evaluation
• Using Whole Document Context in Neural Machine Translation
• Context in Neural Machine Translation: A Review of Models and Evaluations
• Context-Aware Learning for Neural Machine Translation
• Pretrained Language Models for Document-Level Neural Machine Translation
• Linguistic evaluation of German-English Machine Translation using a Test Suite
• Improving Conditioning in Context-Aware Sequence to Sequence Models
• Assessing Crosslingual Discourse Relations in Machine Translation
• Learning Contextualized Sentence Representations for Document-Level Neural Machine Translation
• Improving Context-aware Neural Machine Translation with Target-side Context

Cited by in corpus (3)

• SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task
• N-ODE Transformer: A Depth-Adaptive Variant of the Transformer Using Neural Ordinary Differential Equations
• Deep Sequence Learning for Video Anticipation: From Discrete and Deterministic to Continuous and Stochastic