Unsupervised Neural Machine Translation with Weight Sharing
arXiv:1804.09057
Abstract
Unsupervised neural machine translation (NMT) is a recently proposed approach for machine translation which aims to train the model without using any labeled data. The models proposed for unsupervised NMT often use only one shared encoder to map the pairs of sentences from different languages to a shared-latent space, which is weak in keeping the unique and internal characteristics of each language, such as the style, terminology, and sentence structure. To address this issue, we introduce an extension by utilizing two independent encoders but sharing some partial weights which are responsible for extracting high-level representations of the input sentences. Besides, two different generative adversarial networks (GANs), namely the local GAN and global GAN, are proposed to enhance the cross-language translation. With this new approach, we achieve significant improvements on English-German, English-French and Chinese-to-English translation tasks.
arXiv commentUnsupervised NMT, Accepted by ACL2018, code released
References in corpus (13)
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Convolutional Sequence to Sequence Learning
- Unsupervised Machine Translation Using Monolingual Corpora Only
- DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
- Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
- Minimum Risk Training for Neural Machine Translation
- Neural Machine Translation with Pivot Languages
- Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation
- Deep Neural Machine Translation with Linear Associative Unit
- Learning to Remember Translation History with a Continuous Cache
- Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
- A Correlational Encoder Decoder Architecture for Pivot Based Sequence Generation