Towards Neural Phrase-based Machine Translation
arXiv:1706.05565
Abstract
In this paper, we present Neural Phrase-based Machine Translation (NPMT). Our method explicitly models the phrase structures in output sequences using Sleep-WAke Networks (SWAN), a recently proposed segmentation-based sequence modeling method. To mitigate the monotonic alignment requirement of SWAN, we introduce a new layer to perform (soft) local reordering of input sequences. Different from existing neural machine translation (NMT) approaches, NPMT does not use attention-based decoding mechanisms. Instead, it directly outputs phrases in a sequential order and can decode in linear time. Our experiments show that NPMT achieves superior performances on IWSLT 2014 German-English/English-German and IWSLT 2015 English-Vietnamese machine translation tasks compared with strong NMT baselines. We also observe that our method produces meaningful phrases in output languages.
in International Conference on Learning Representations (ICLR) 2018
Cited by in corpus (10)
- Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
- Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction
- MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning
- Towards Binary-Valued Gates for Robust LSTM Training
- Structured Reordering for Modeling Latent Alignments in Sequence Transduction
- Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning
- Efficient Bidirectional Neural Machine Translation
- Neural Machine Translation: A Review and Survey
- Alleviate Exposure Bias in Sequence Prediction \\ with Recurrent Neural Networks
- Sequence-to-Sequence Learning with Latent Neural Grammars