Explaining and Generalizing Back-Translation through Wake-Sleep

Abstract

Back-translation has become a commonly employed heuristic for semi-supervised neural machine translation. The technique is both straightforward to apply and has led to state-of-the-art results. In this work, we offer a principled interpretation of back-translation as approximate inference in a generative model of bitext and show how the standard implementation of back-translation corresponds to a single iteration of the wake-sleep algorithm in our proposed model. Moreover, this interpretation suggests a natural iterative generalization, which we demonstrate leads to further improvement of up to 1.6 BLEU.

References in corpus (4)

• Sequence to Sequence Learning with Neural Networks
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Dual Learning for Machine Translation
• An Actor-Critic Algorithm for Sequence Prediction

Cited by in corpus (5)

• Understanding Back-Translation at Scale
• A Multilingual Parallel Corpora Collection Effort for Indian Languages
• Revisiting Iterative Back-Translation from the Perspective of Compositional Generalization
• Diversifying Dialogue Generation with Non-Conversational Text
• Iterative Batch Back-Translation for Neural Machine Translation: A Conceptual Model