You May Not Need Attention
arXiv:1810.13409
Abstract
In NMT, how far can we get without attention and without separate encoding and decoding? To answer that question, we introduce a recurrent neural translation model that does not use attention and does not have a separate encoder and decoder. Our eager translation model is low-latency, writing target tokens as soon as it reads the first source token, and uses constant memory during decoding. It performs on par with the standard attention-based model of Bahdanau et al. (2014), and better on long sentences.
References in corpus (8)
- Sequence to Sequence Learning with Neural Networks
- Convolutional Sequence to Sequence Learning
- Recurrent Neural Network Regularization
- Regularizing and Optimizing LSTM Language Models
- Neural Machine Translation in Linear Time
- Grid Long Short-Term Memory
- Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
- Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction
Cited by in corpus (7)
- DuTongChuan: Context-aware Translation Model for Simultaneous Interpreting
- Retrieving Sequential Information for Non-Autoregressive Neural Machine Translation
- STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework
- Faster Re-translation Using Non-Autoregressive Model For Simultaneous Neural Machine Translation
- Simultaneous Translation with Flexible Policy via Restricted Imitation Learning
- Infusing Future Information into Monotonic Attention Through Language Models
- Full-Sentence Models Perform Better in Simultaneous Translation Using the Information Enhanced Decoding Strategy