Dynamic Past and Future for Neural Machine Translation
arXiv:1904.09646
Abstract
Previous studies have shown that neural machine translation (NMT) models can benefit from explicitly modeling translated (Past) and untranslated (Future) to groups of translated and untranslated contents through parts-to-wholes assignment. The assignment is learned through a novel variant of routing-by-agreement mechanism (Sabour et al., 2017), namely {\em Guided Dynamic Routing}, where the translating status at each decoding step {\em guides} the routing process to assign each source word to its associated group (i.e., translated or untranslated content) represented by a capsule, enabling translation to be made from holistic context. Experiments show that our approach achieves substantial improvements over both RNMT and Transformer by producing more adequate translations. Extensive analysis demonstrates that our method is highly interpretable, which is able to recognize the translated and untranslated contents as expected.
Camera-ready version. Accepted to EMNLP 2019 as a long paper
References in corpus (9)
- Sequence to Sequence Learning with Neural Networks
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Convolutional Sequence to Sequence Learning
- Investigating Capsule Networks with Dynamic Routing for Text Classification
- Twin Networks: Matching the Future for Sequence Generation
- Learning to Decode for Future Success
- Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement
- Deconvolution-Based Global Decoding for Neural Machine Translation
- Correct-and-Memorize: Learning to Translate from Interactive Revisions