On the Properties of Neural Machine Translation: Encoder-Decoder Approaches

Abstract

Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates a correct translation from this representation. In this paper, we focus on analyzing the properties of the neural machine translation using two models; RNN Encoder--Decoder and a newly proposed gated recursive convolutional neural network. We show that the neural machine translation performs relatively well on short sentences without unknown words, but its performance degrades rapidly as the length of the sentence and the number of unknown words increase. Furthermore, we find that the proposed gated recursive convolutional network learns a grammatical structure of a sentence automatically.

Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (SSST-8)

References in corpus (1)

• Sequence Transduction with Recurrent Neural Networks

Cited by in corpus (64)

• Comparative Study of CNN and RNN for Natural Language Processing
• VulDeePecker: A Deep Learning-Based System for Vulnerability Detection
• Skip-Thought Vectors
• DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset
• Depthwise Separable Convolutions for Neural Machine Translation
• Learning What and Where to Draw
• Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Fast Domain Adaptation for Neural Machine Translation
• Full-Capacity Unitary Recurrent Neural Networks
• Compositional Vector Space Models for Knowledge Base Completion
• ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA
• Convolutional Recurrent Neural Networks for Music Classification
• Tensor-Train Recurrent Neural Networks for Video Classification
• Challenges in Data-to-Document Generation
• Revisiting the Effectiveness of Off-the-shelf Temporal Modeling Approaches for Large-scale Video Classification
• Embedding Word Similarity with Neural Machine Translation
• Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering
• Linguistic Knowledge as Memory for Recurrent Neural Networks
• Learning Deep Neural Networks for Vehicle Re-ID with Visual-spatio-temporal Path Proposals
• Visualizing and Understanding Curriculum Learning for Long Short-Term Memory Networks
• Knowledge as a Teacher: Knowledge-Guided Structural Attention Networks
• Understanding Hidden Memories of Recurrent Neural Networks
• Unsupervised temporal context learning using convolutional neural networks for laparoscopic workflow analysis
• Online Learning for Neural Machine Translation Post-editing
• Learning to Explain Non-Standard English Words and Phrases
• Learning Robust Dialog Policies in Noisy Environments
• Grounded Recurrent Neural Networks
• Memory-augmented Neural Machine Translation
• Neural Networks Models for Entity Discovery and Linking
• Toward Goal-Driven Neural Network Models for the Rodent Whisker-Trigeminal System
• Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Records
• Improving the Performance of Neural Machine Translation Involving Morphologically Rich Languages
• Improved Regularization Techniques for End-to-End Speech Recognition
• Code Attention: Translating Code to Comments by Exploiting Domain Features
• Identifying Harm Events in Clinical Care through Medical Narratives
• A Flexible Approach to Automated RNN Architecture Generation
• Graph Clustering with Dynamic Embedding
• Predicting Movie Genres Based on Plot Summaries
• Capturing Long-range Contextual Dependencies with Memory-enhanced Conditional Random Fields
• Toward a full-scale neural machine translation in production: the Booking.com use case
• Face Parsing via Recurrent Propagation
• Recurrent Latent Variable Networks for Session-Based Recommendation
• Machine Translation at Booking.com: Journey and Lessons Learned
• Improving End-to-End Speech Recognition with Policy Learning
• Log-Linear RNNs: Towards Recurrent Neural Networks with Flexible Prior Knowledge
• Getting deep recommenders fit: Bloom embeddings for sparse binary input/output networks
• Revisiting the Design Issues of Local Models for Japanese Predicate-Argument Structure Analysis
• Bidirectional Beam Search: Forward-Backward Inference in Neural Sequence Models for Fill-in-the-Blank Image Captioning
• An Efficient Character-Level Neural Machine Translation
• Improving historical spelling normalization with bi-directional LSTMs and multi-task learning
• Scene Labeling using Gated Recurrent Units with Explicit Long Range Conditioning
• Autoencoder Regularized Network For Driving Style Representation Learning
• Memory Visualization for Gated Recurrent Neural Networks in Speech Recognition
• Analogs of Linguistic Structure in Deep Representations
• Empirical Evaluation of RNN Architectures on Sentence Classification Task
• Interpreting the Syntactic and Social Elements of the Tweet Representations via Elementary Property Prediction Tasks
• Enhanced Neural Machine Translation by Learning from Draft
• Towards Music Captioning: Generating Music Playlist Descriptions
• Rotational Unit of Memory
• Dynamic Graph Generation Network: Generating Relational Knowledge from Diagrams
• GeoSeq2Seq: Information Geometric Sequence-to-Sequence Networks
• Scalable Machine Translation in Memory Constrained Environments
• Plan, Attend, Generate: Character-level Neural Machine Translation with Planning in the Decoder