Neighbors Are Not Strangers: Improving Non-Autoregressive Translation under Low-Frequency Lexical Constraints
arXiv:2204.13355 · doi:10.18653/v1/2022.naacl-main.424
Abstract
However, current autoregressive approaches suffer from high latency. In this paper, we focus on non-autoregressive translation (NAT) for this problem for its efficiency advantage. We identify that current constrained NAT models, which are based on iterative editing, do not handle low-frequency constraints well. To this end, we propose a plug-in algorithm for this line of work, i.e., Aligned Constrained Training (ACT), which alleviates this problem by familiarizing the model with the source-side context of the constraints. Experiments on the general and domain datasets show that our model improves over the backbone constrained NAT model in constraint preservation and translation quality, especially for rare constraints.
arXiv commentAccepted to NAACL 2022
References in corpus (10)
- Adam: A Method for Stochastic Optimization
- Neural Machine Translation by Jointly Learning to Align and Translate
- Non-Autoregressive Neural Machine Translation
- Levenshtein Transformer
- Fast Decoding in Sequence Models using Discrete Latent Variables
- SYSTRAN's Pure Neural Machine Translation Systems
- Non-Autoregressive Machine Translation with Disentangled Context Transformer
- Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation
- Understanding and Improving Lexical Choice in Non-Autoregressive Translation
- Incorporating BERT into Parallel Sequence Decoding with Adapters