A Simple Baseline to Semi-Supervised Domain Adaptation for Machine Translation

Abstract

State-of-the-art neural machine translation (NMT) systems are data-hungry and perform poorly on new domains with no supervised data. As data collection is expensive and infeasible in many cases, domain adaptation methods are needed. In this work, we propose a simple but effect approach to the semi-supervised domain adaptation scenario of NMT, where the aim is to improve the performance of a translation model on the target domain consisting of only non-parallel data with the help of supervised source domain data. This approach iteratively trains a Transformer-based NMT model via three training objectives: language modeling, back-translation, and supervised translation. We evaluate this method on two adaptation settings: adaptation between specific domains and adaptation from a general domain to specific domains, and on two language pairs: German to English and Romanian to English. With substantial performance improvement achieved---up to +19.31 BLEU over the strongest baseline, and +47.69 BLEU improvement over the unadapted model---we present this method as a simple but tough-to-beat baseline in the field of semi-supervised domain adaptation for NMT.

arXiv commentUnder review

References in corpus (3)

• Dual Learning for Machine Translation
• Phrase-Based & Neural Unsupervised Machine Translation
• Fast Domain Adaptation for Neural Machine Translation

Cited by in corpus (6)

• CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
• Bridged-GNN: Knowledge Bridge Learning for Effective Knowledge Transfer
• Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings
• Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters
• Rapid Domain Adaptation for Machine Translation with Monolingual Data
• Towards Zero and Few-shot Knowledge-seeking Turn Detection in Task-orientated Dialogue Systems