Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Information
arXiv:1911.02215
Abstract
Non-autoregressive neural machine translation (NAT) generates each target word in parallel and has achieved promising inference acceleration. However, existing NAT models still have a big gap in translation quality compared to autoregressive neural machine translation models due to the enormous decoding space. To address this problem, we propose a novel NAT framework named ReorderNAT which explicitly models the reordering information in the decoding procedure. We further introduce deterministic and non-deterministic decoding strategies that utilize reordering information to narrow the decoding search space in our proposed ReorderNAT. Experimental results on various widely-used datasets show that our proposed model achieves better performance compared to existing NAT models, and even achieves comparable translation quality as autoregressive translation models with a significant speedup.
arXiv commentAccepted by AAAI 2021
References in corpus (16)
- Sequence to Sequence Learning with Neural Networks
- Non-Autoregressive Neural Machine Translation
- Fast Decoding in Sequence Models using Discrete Latent Variables
- Insertion Transformer: Flexible Sequence Generation via Insertion Operations
- Non-Autoregressive Machine Translation with Disentangled Context Transformer
- Syntactically Supervised Transformers for Faster Neural Machine Translation
- Semi-Autoregressive Training Improves Mask-Predict Decoding
- Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translation
- Non-autoregressive Transformer by Position Learning
- An EM Approach to Non-autoregressive Conditional Sequence Generation
- Retrieving Sequential Information for Non-Autoregressive Neural Machine Translation
- LAVA NAT: A Non-Autoregressive Translation Model with Look-Around Decoding and Vocabulary Attention
- Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation
- Improving Non-autoregressive Neural Machine Translation with Monolingual Data
- Task-Level Curriculum Learning for Non-Autoregressive Neural Machine Translation
- Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translation