Using Monolingual Data in Neural Machine Translation: a Systematic Study

Abstract

Neural Machine Translation (MT) has radically changed the way systems are developed. A major difference with the previous generation (Phrase-Based MT) is the way monolingual target data, which often abounds, is used in these two paradigms. While Phrase-Based MT can seamlessly integrate very large language models trained on billions of sentences, the best option for Neural MT developers seems to be the generation of artificial parallel data through \textsl{back-translation} - a technique that fails to fully take advantage of existing datasets. In this paper, we conduct a systematic study of back-translation, comparing alternative uses of monolingual data, as well as multiple data generation procedures. Our findings confirm that back-translation is very effective and give new explanations as to why this is the case. We also introduce new data simulation techniques that are almost as effective, yet much cheaper to implement.

Published in the Proceedings of the Third Conference on Machine Translation (Research Papers), 2018

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Dual Learning for Machine Translation
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• Unsupervised Machine Translation Using Monolingual Corpora Only
• Building a Neural Machine Translation System Using Only Synthetic Parallel Data
• Neural Machine Translation from Simplified Translations

Cited by in corpus (2)

• Iterative Batch Back-Translation for Neural Machine Translation: A Conceptual Model
• Empirical Analysis of Korean Public AI Hub Parallel Corpora and in-depth Analysis using LIWC