On the Properties of Neural Machine Translation: Encoder-Decoder Approaches

Abstract

Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates a correct translation from this representation. In this paper, we focus on analyzing the properties of the neural machine translation using two models; RNN Encoder--Decoder and a newly proposed gated recursive convolutional neural network. We show that the neural machine translation performs relatively well on short sentences without unknown words, but its performance degrades rapidly as the length of the sentence and the number of unknown words increase. Furthermore, we find that the proposed gated recursive convolutional network learns a grammatical structure of a sentence automatically.

Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (SSST-8)

References in corpus (1)

• Sequence Transduction with Recurrent Neural Networks

Cited by in corpus (135)

• Comparative Study of CNN and RNN for Natural Language Processing
• VulDeePecker: A Deep Learning-Based System for Vulnerability Detection
• Skip-Thought Vectors
• DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset
• Working Memory Connections for LSTM
• Depthwise Separable Convolutions for Neural Machine Translation
• Learning What and Where to Draw
• Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Fast Domain Adaptation for Neural Machine Translation
• Social Interactions for Autonomous Driving: A Review and Perspectives
• Analyzing Input and Output Representations for Speech-Driven Gesture Generation
• Full-Capacity Unitary Recurrent Neural Networks
• Gait Recognition Based on Deep Learning: A Survey
• FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network
• Exploring Interpretable LSTM Neural Networks over Multi-Variable Data
• Compositional Vector Space Models for Knowledge Base Completion
• ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA
• Convolutional Recurrent Neural Networks for Music Classification
• Tensor-Train Recurrent Neural Networks for Video Classification
• Challenges in Data-to-Document Generation
• Revisiting the Effectiveness of Off-the-shelf Temporal Modeling Approaches for Large-scale Video Classification
• Embedding Word Similarity with Neural Machine Translation
• Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering
• DeepSpline: Data-Driven Reconstruction of Parametric Curves and Surfaces
• Linguistic Knowledge as Memory for Recurrent Neural Networks
• Learning Deep Neural Networks for Vehicle Re-ID with Visual-spatio-temporal Path Proposals
• Visualizing and Understanding Curriculum Learning for Long Short-Term Memory Networks
• Knowledge as a Teacher: Knowledge-Guided Structural Attention Networks
• Understanding Hidden Memories of Recurrent Neural Networks
• Transcribing Content from Structural Images with Spotlight Mechanism
• Unsupervised temporal context learning using convolutional neural networks for laparoscopic workflow analysis
• Online Learning for Neural Machine Translation Post-editing
• Learning to Explain Non-Standard English Words and Phrases
• Emotion-Aware Transformer Encoder for Empathetic Dialogue Generation
• Grounded Recurrent Neural Networks
• Learning Robust Dialog Policies in Noisy Environments
• Memory-augmented Neural Machine Translation
• A Novel Ensemble Deep Learning Model for Stock Prediction Based on Stock Prices and News
• Neural Networks Models for Entity Discovery and Linking
• A Survey on Recent Advances in Sequence Labeling from Deep Learning Models
• Toward Goal-Driven Neural Network Models for the Rodent Whisker-Trigeminal System
• Rare Disease Detection by Sequence Modeling with Generative Adversarial Networks
• Predictive analysis of Bitcoin price considering social sentiments
• Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Records
• Augmenting Non-Collaborative Dialog Systems with Explicit Semantic and Strategic Dialog History
• Neural Machine Translation For Paraphrase Generation
• Code Attention: Translating Code to Comments by Exploiting Domain Features
• EdgeRec: Recommender System on Edge in Mobile Taobao
• Improving the Performance of Neural Machine Translation Involving Morphologically Rich Languages
• Improved Regularization Techniques for End-to-End Speech Recognition
• A Flexible Approach to Automated RNN Architecture Generation
• Identifying Harm Events in Clinical Care through Medical Narratives
• Graph Clustering with Dynamic Embedding
• Predicting Movie Genres Based on Plot Summaries
• SMT vs NMT: A Comparison over Hindi & Bengali Simple Sentences
• Deep Technology Tracing for High-tech Companies
• Capturing Long-range Contextual Dependencies with Memory-enhanced Conditional Random Fields
• Multimodal Trajectory Prediction via Topological Invariance for Navigation at Uncontrolled Intersections
• DTCA: Decision Tree-based Co-Attention Networks for Explainable Claim Verification
• From Words to Sentences: A Progressive Learning Approach for Zero-resource Machine Translation with Visual Pivots
• Context, Attention and Audio Feature Explorations for Audio Visual Scene-Aware Dialog
• Literature review on vulnerability detection using NLP technology
• Face Parsing via Recurrent Propagation
• Toward a full-scale neural machine translation in production: the Booking.com use case
• Machine Translation at Booking.com: Journey and Lessons Learned
• Recurrent Latent Variable Networks for Session-Based Recommendation
• Towards Neural Decompilation
• CTCModel: a Keras Model for Connectionist Temporal Classification
• Temporal Context Aggregation for Video Retrieval with Contrastive Learning
• Automated essay scoring using efficient transformer-based language models
• Multi-task Learning based Pre-trained Language Model for Code Completion
• Log-Linear RNNs: Towards Recurrent Neural Networks with Flexible Prior Knowledge
• Divide-and-Conquer Adversarial Learning for High-Resolution Image and Video Enhancement
• Improving End-to-End Speech Recognition with Policy Learning
• Getting deep recommenders fit: Bloom embeddings for sparse binary input/output networks
• A Survey on Deep Learning for Neuroimaging-based Brain Disorder Analysis
• Time-series Insights into the Process of Passing or Failing Online University Courses using Neural-Induced Interpretable Student States
• Revisiting the Design Issues of Local Models for Japanese Predicate-Argument Structure Analysis
• MixSeq: Connecting Macroscopic Time Series Forecasting with Microscopic Time Series Data
• An Efficient Character-Level Neural Machine Translation
• Bidirectional Beam Search: Forward-Backward Inference in Neural Sequence Models for Fill-in-the-Blank Image Captioning
• BanglaBait: Semi-Supervised Adversarial Approach for Clickbait Detection on Bangla Clickbait Dataset
• External Correlates of Adult Digital Problem-Solving Behavior: Log Data Analysis of a Large-Scale Assessment
• Explainable Multivariate Time Series Classification: A Deep Neural Network Which Learns To Attend To Important Variables As Well As Informative Time Intervals
• Scene Labeling using Gated Recurrent Units with Explicit Long Range Conditioning
• Tensorized Transformer for Dynamical Systems Modeling
• Improving historical spelling normalization with bi-directional LSTMs and multi-task learning
• Optimal Mini-Batch Size Selection for Fast Gradient Descent
• Autoencoder Regularized Network For Driving Style Representation Learning
• Interpreting the Syntactic and Social Elements of the Tweet Representations via Elementary Property Prediction Tasks
• Empirical Evaluation of RNN Architectures on Sentence Classification Task
• Polyphone Disambiguation for Mandarin Chinese Using Conditional Neural Network with Multi-level Embedding Features
• Enhancing Salient Object Segmentation Through Attention
• CryptoAnalytics: Cryptocoins Price Forecasting with Machine Learning Techniques
• Analogs of Linguistic Structure in Deep Representations
• Spatial-Temporal Tensor Graph Convolutional Network for Traffic Prediction
• Deep Animation Video Interpolation in the Wild
• Mimic and Conquer: Heterogeneous Tree Structure Distillation for Syntactic NLP
• Memory Visualization for Gated Recurrent Neural Networks in Speech Recognition
• CNNs, LSTMs, and Attention Networks for Pathology Detection in Medical Data
• Extending a model for ontology-based Arabic-English machine translation
• Towards Music Captioning: Generating Music Playlist Descriptions
• Multilevel Text Normalization with Sequence-to-Sequence Networks and Multisource Learning
• Enhanced Neural Machine Translation by Learning from Draft
• Rotational Unit of Memory
• Eigenvalue Normalized Recurrent Neural Networks for Short Term Memory
• Patching as Translation: the Data and the Metaphor
• Self-Supervised Learning and Prediction of Microstructure Evolution with Recurrent Neural Networks
• EDSL: An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition
• A Hierarchical Deep Convolutional Neural Network and Gated Recurrent Unit Framework for Structural Damage Detection
• Dynamic Graph Generation Network: Generating Relational Knowledge from Diagrams
• Let's See Clearly: Contaminant Artifact Removal for Moving Cameras
• An Optimized and Energy-Efficient Parallel Implementation of Non-Iteratively Trained Recurrent Neural Networks
• Memory and attention in deep learning
• Learning Dynamical Systems from Noisy Sensor Measurements using Multiple Shooting
• A Variational Auto-Encoder for Reservoir Monitoring
• Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning
• A Deep Structural Model for Analyzing Correlated Multivariate Time Series
• GeoSeq2Seq: Information Geometric Sequence-to-Sequence Networks
• Victim or Perpetrator? Analysis of Violent Characters Portrayals from Movie Scripts
• Latent Variable Algorithms for Multimodal Learning and Sensor Fusion
• Using GGNN to recommend log statement level
• Plan, Attend, Generate: Character-level Neural Machine Translation with Planning in the Decoder
• An End-to-End Approach to Automatic Speech Assessment for Cantonese-speaking People with Aphasia
• Applying recent advances in Visual Question Answering to Record Linkage
• Combining Word and Character Vector Representation on Neural Machine Translation
• Enhancing Model Robustness By Incorporating Adversarial Knowledge Into Semantic Representation
• Data-Efficient Methods for Dialogue Systems
• Shapley Homology: Topological Analysis of Sample Influence for Neural Networks
• Scalable Machine Translation in Memory Constrained Environments
• Multi-Step Chord Sequence Prediction Based on Aggregated Multi-Scale Encoder-Decoder Network
• Efficient Global String Kernel with Random Features: Beyond Counting Substructures
• Structured Prediction in NLP -- A survey
• Maintaining Common Ground in Dynamic Environments