Neural Machine Translation with Reconstruction

Abstract

Although end-to-end Neural Machine Translation (NMT) has achieved remarkable progress in the past two years, it suffers from a major drawback: translations generated by NMT systems often lack of adequacy. It has been widely observed that NMT tends to repeatedly translate some source words while mistakenly ignoring other words. To alleviate this problem, we propose a novel encoder-decoder-reconstructor framework for NMT. The reconstructor, incorporated into the NMT model, manages to reconstruct the input source sentence from the hidden layer of the output target sentence, to ensure that the information in the source side is transformed to the target side as much as possible. Experiments show that the proposed framework significantly improves the adequacy of NMT output and achieves superior translation result over state-of-the-art NMT and statistical MT systems.

Accepted by AAAI 2017

References in corpus (7)

• Improving neural networks by preventing co-adaptation of feature detectors
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Effective Approaches to Attention-based Neural Machine Translation
• On Using Monolingual Corpora in Neural Machine Translation
• Modeling Coverage for Neural Machine Translation
• Mutual Information and Diverse Decoding Improve Neural Machine Translation
• Syntactically Guided Neural Machine Translation

Cited by in corpus (27)

• Adversarial Neural Machine Translation
• Massive Exploration of Neural Machine Translation Architectures
• Reconstruction Network for Video Captioning
• Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities
• Prior-Knowledge and Attention-based Meta-Learning for Few-Shot Learning
• Exploiting Cross-Sentence Context for Neural Machine Translation
• Latent Variable Dialogue Models and their Diversity
• Modeling Source Syntax for Neural Machine Translation
• The Neural Noisy Channel
• Non-Autoregressive Machine Translation with Auxiliary Regularization
• Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation
• Projection Based Weight Normalization for Deep Neural Networks
• Trainable Greedy Decoding for Neural Machine Translation
• Learning to Remember Translation History with a Continuous Cache
• Source-side Prediction for Neural Headline Generation
• Translating Phrases in Neural Machine Translation
• Neural Machine Translation Advised by Statistical Machine Translation
• Chunk-Based Bi-Scale Decoder for Neural Machine Translation
• A Stable and Effective Learning Strategy for Trainable Greedy Decoding
• Learning to Jointly Translate and Predict Dropped Pronouns with a Shared Reconstruction Mechanism
• Modeling Past and Future for Neural Machine Translation
• English-Japanese Neural Machine Translation with Encoder-Decoder-Reconstructor
• Language-Independent Representor for Neural Machine Translation
• Online Back-Parsing for AMR-to-Text Generation
• Exploring Recombination for Efficient Decoding of Neural Machine Translation
• SU-RUG at the CoNLL-SIGMORPHON 2017 shared task: Morphological Inflection with Attentional Sequence-to-Sequence Models
• A Technical Report: BUT Speech Translation Systems