Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Abstract

In this paper, we propose a novel neural network model called RNN Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixed-length vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of the proposed model are jointly trained to maximize the conditional probability of a target sequence given a source sequence. The performance of a statistical machine translation system is empirically found to improve by using the conditional probabilities of phrase pairs computed by the RNN Encoder-Decoder as an additional feature in the existing log-linear model. Qualitatively, we show that the proposed model learns a semantically and syntactically meaningful representation of linguistic phrases.

EMNLP 2014

References in corpus (1)

• ADADELTA: An Adaptive Learning Rate Method

Cited by in corpus (10)

• Sequence to Sequence Learning with Neural Networks
• On Using Monolingual Corpora in Neural Machine Translation
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• Learning Stochastic Recurrent Networks
• Variational Recurrent Auto-Encoders
• Encoding Source Language with Convolutional Neural Network for Machine Translation
• Embedding Word Similarity with Neural Machine Translation
• Context-Dependent Translation Selection Using Convolutional Neural Network
• The NLP Engine: A Universal Turing Machine for NLP
• A Hierarchical Recurrent Encoder-Decoder For Generative Context-Aware Query Suggestion