Multilingual Neural Machine Translation with Task-Specific Attention

Abstract

Multilingual machine translation addresses the task of translating between multiple source and target languages. We propose task-specific attention models, a simple but effective technique for improving the quality of sequence-to-sequence neural multilingual translation. Our approach seeks to retain as much of the parameter sharing generalization of NMT models as possible, while still allowing for language-specific specialization of the attention model to a particular language-pair or task. Our experiments on four languages of the Europarl corpus show that using a target-specific model of attention provides consistent gains in translation quality for all possible translation directions, compared to a model in which all parameters are shared. We observe improved translation quality even in the (extreme) low-resource zero-shot translation directions for which the model never saw explicitly paired parallel data.

arXiv commentCOLING 2018

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Neural Machine Translation by Jointly Learning to Align and Translate
• Sequence to Sequence Learning with Neural Networks
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• DyNet: The Dynamic Neural Network Toolkit
• Cross-lingual Models of Word Embeddings: An Empirical Comparison

Cited by in corpus (8)

• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• The Missing Ingredient in Zero-Shot Neural Machine Translation
• Multilingual NMT with a language-independent attention bridge
• A Brief Survey of Multilingual Neural Machine Translation
• A Survey on Low-Resource Neural Machine Translation
• A Survey on Green Deep Learning
• Improving Multilingual Translation by Representation and Gradient Regularization
• Self-Learning for Zero Shot Neural Machine Translation