Neural Machine Translation with Supervised Attention
arXiv:1609.04186
Abstract
The attention mechanisim is appealing for neural machine translation, since it is able to dynam- ically encode a source sentence by generating a alignment between a target word and source words. Unfortunately, it has been proved to be worse than conventional alignment models in aligment accuracy. In this paper, we analyze and explain this issue from the point view of re- ordering, and propose a supervised attention which is learned with guidance from conventional alignment models. Experiments on two Chinese-to-English translation tasks show that the super- vised attention mechanism yields better alignments leading to substantial gains over the standard attention based NMT.
This paper was submitted into COLING2016 on July 10, and it is under review
References in corpus (6)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- ADADELTA: An Adaptive Learning Rate Method
- Effective Approaches to Attention-based Neural Machine Translation
- Modeling Coverage for Neural Machine Translation
- Implicit Distortion and Fertility Models for Attention-based Encoder-Decoder NMT Model
- Incorporating Discrete Translation Lexicons into Neural Machine Translation
Cited by in corpus (11)
- An Empirical Study of Spatial Attention Mechanisms in Deep Networks
- Bridging Neural Machine Translation and Bilingual Dictionaries
- Learning to Generate Corrective Patches using Neural Machine Translation
- Translating Phrases in Neural Machine Translation
- GASL: Guided Attention for Sparsity Learning in Deep Neural Networks
- Deriving Machine Attention from Human Rationales
- Towards Neural Machine Translation with Partially Aligned Corpora
- Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings
- Enriched Annotations for Tumor Attribute Classification from Pathology Reports with Limited Labeled Data
- Regularized Context Gates on Transformer for Machine Translation
- Prior Attention for Style-aware Sequence-to-Sequence Models