Iterative Refinement for Machine Translation

Abstract

Existing machine translation decoding algorithms generate translations in a strictly monotonic fashion and never revisit previous decisions. As a result, earlier mistakes cannot be corrected at a later stage. In this paper, we present a translation scheme that starts from an initial guess and then makes iterative improvements that may revisit previous decisions. We parameterize our model as a convolutional neural network that predicts discrete substitutions to an existing translation based on an attention mechanism over both the source sentence as well as the current translation output. By making less than one modification per sentence, we improve the output of a phrase-based translation system by up to 0.4 BLEU on WMT15 German-English translation.

Presented as a poster at BayLearn 2017

References in corpus (1)

• Effective Approaches to Attention-based Neural Machine Translation

Cited by in corpus (9)

• Retrieve and Refine: Improved Sequence Generation Models For Dialogue
• Neural Networks for Modeling Source Code Edits
• XL-Editor: Post-editing Sentences with XLNet
• Progressive Generation of Long Text with Pretrained Language Models
• PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation
• Enhanced Neural Machine Translation by Learning from Draft
• Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translation
• Distilling Translations with Visual Awareness
• Semantic Role Labeling with Iterative Structure Refinement