Neural Machine Translation: A Review and Survey

Abstract

The field of machine translation (MT), the automatic translation of written text from one natural language into another, has experienced a major paradigm shift in recent years. Statistical MT, which mainly relies on various count-based models and which used to dominate MT research for decades, has largely been superseded by neural machine translation (NMT), which tackles translation with a single neural network. In this work we will trace back the origins of modern NMT architectures to word and sentence embeddings and earlier examples of the encoder-decoder network family. We will conclude with a survey of recent trends in the field.

arXiv commentExtended version of "Neural Machine Translation: A Review" accepted by the Journal of Artificial Intelligence Research (JAIR)

References in corpus (164)

• Efficient Estimation of Word Representations in Vector Space
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Distilling the Knowledge in a Neural Network
• Sequence to Sequence Learning with Neural Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Overcoming catastrophic forgetting in neural networks
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• ADADELTA: An Adaptive Learning Rate Method
• Natural Language Processing (almost) from Scratch
• Neural Architecture Search with Reinforcement Learning
• On the difficulty of training Recurrent Neural Networks
• Towards A Rigorous Science of Interpretable Machine Learning
• Methods for Interpreting and Understanding Deep Neural Networks
• Teaching Machines to Read and Comprehend
• Convolutional Sequence to Sequence Learning
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Pixel Recurrent Neural Networks
• A Structured Self-attentive Sentence Embedding
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Exploiting Similarities among Languages for Machine Translation
• A Deep Reinforced Model for Abstractive Summarization
• Theano: new features and speed improvements
• Character-Aware Neural Language Models
• Recurrent Models of Visual Attention
• Sequence Level Training with Recurrent Neural Networks
• Visualizing and Understanding Recurrent Networks
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Multiple Object Recognition with Visual Attention
• End-To-End Memory Networks
• To prune, or not to prune: exploring the efficacy of pruning for model compression
• Dual Learning for Machine Translation
• Don't Decay the Learning Rate, Increase the Batch Size
• Achieving Human Parity on Automatic Chinese to English News Translation
• On Using Monolingual Corpora in Neural Machine Translation
• Non-Autoregressive Neural Machine Translation
• Attention is not Explanation
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• Synthetic and Natural Noise Both Break Neural Machine Translation
• Regularizing Neural Networks by Penalizing Confident Output Distributions
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Pay Less Attention with Lightweight and Dynamic Convolutions
• Neural Machine Translation in Linear Time
• A Fast and Simple Algorithm for Training Neural Probabilistic Language Models
• Assessing BERT's Syntactic Abilities
• Training Tips for the Transformer Model
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• Pathologies of Neural Models Make Interpretations Difficult
• Depthwise Separable Convolutions for Neural Machine Translation
• Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
• Online and Linear-Time Attention by Enforcing Monotonic Alignments
• Better Mixing via Deep Representations
• Character-based Neural Machine Translation
• The Evolved Transformer
• A Survey of Domain Adaptation for Neural Machine Translation
• Unsupervised Machine Translation Using Monolingual Corpora Only
• Sockeye: A Toolkit for Neural Machine Translation
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
• Fast Decoding in Sequence Models using Discrete Latent Variables
• Fast Domain Adaptation for Neural Machine Translation
• Insertion Transformer: Flexible Sequence Generation via Insertion Operations
• Modeling Coverage for Neural Machine Translation
• An Empirical Model of Large-Batch Training
• Weighted Transformer Network for Machine Translation
• Learning to Transduce with Unbounded Memory
• Investigating Backtranslation in Neural Machine Translation
• Can neural machine translation do simultaneous translation?
• Does Neural Machine Translation Benefit from Larger Context?
• Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
• Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation
• Understanding Back-Translation at Scale
• Multimodal Pivots for Image Caption Translation
• Mutual Information and Diverse Decoding Improve Neural Machine Translation
• A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translation
• Unsupervised Neural Machine Translation
• THUMT: An Open Source Toolkit for Neural Machine Translation
• Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs
• Adding Interpretable Attention to Neural Translation Models Improves Word Alignment
• Language GANs Falling Short
• SYSTRAN's Pure Neural Machine Translation Systems
• No Training Required: Exploring Random Encoders for Sentence Classification
• On the Computational Efficiency of Training Neural Networks
• Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation
• Adversarial Neural Machine Translation
• Multilingual Image Description with Neural Sequence Models
• Distance-based Self-Attention Network for Natural Language Inference
• Pre-Translation for Neural Machine Translation
• Multi-task Sequence to Sequence Learning
• Notes on Noise Contrastive Estimation and Negative Sampling
• Ensemble Distillation for Neural Machine Translation
• Non-Monotonic Sequential Text Generation
• Lattice-Based Recurrent Neural Network Encoders for Neural Machine Translation
• Syntax-Directed Attention for Neural Machine Translation
• XNMT: The eXtensible Neural Machine Translation Toolkit
• RNN Approaches to Text Normalization: A Challenge
• Neural Reranking Improves Subjective Quality of Machine Translation: NAIST at WAT2015
• Neural Machine Translation with External Phrase Memory
• Hierarchical Memory Networks
• Neural Machine Translation Training in a Multi-Domain Scenario
• Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model
• Language Models with Transformers
• Bridging Neural Machine Translation and Bilingual Dictionaries
• Cross-lingual Models of Word Embeddings: An Empirical Comparison
• Effective Strategies in Zero-Shot Neural Machine Translation
• Generative Neural Machine Translation
• Calibration of Encoder Decoder Models for Neural Machine Translation
• Implicit Distortion and Fertility Models for Attention-based Encoder-Decoder NMT Model
• Vocabulary Selection Strategies for Neural Machine Translation
• End-to-End Speech Translation with Knowledge Distillation
• Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder
• Dynamic Evaluation of Transformer Language Models
• Identifying and Controlling Important Neurons in Neural Machine Translation
• Neural machine translation for low-resource languages
• Text normalization using memory augmented neural networks
• A GRU-Gated Attention Model for Neural Machine Translation
• Insertion-based Decoding with automatically Inferred Generation Order
• Assessing the Tolerance of Neural Machine Translation Systems Against Speech Recognition Errors
• Boosting Neural Machine Translation
• Multi-Domain Neural Machine Translation
• Towards Neural Phrase-based Machine Translation
• NoiseOut: A Simple Way to Prune Neural Networks
• Context in Neural Machine Translation: A Review of Models and Evaluations
• Neutron: An Implementation of the Transformer Translation Model and its Variants
• The Neural Noisy Channel
• QCRI Machine Translation Systems for IWSLT 16
• Confidence through Attention
• Towards Robust Neural Machine Translation
• Self-Attentive Residual Decoder for Neural Machine Translation
• Unsupervised Neural Machine Translation with SMT as Posterior Regularization
• Neural System Combination for Machine Translation
• Translation of Patent Sentences with a Large Vocabulary of Technical Terms Using Neural Machine Translation
• Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input
• Competence-based Curriculum Learning for Neural Machine Translation
• A Fully Differentiable Beam Search Decoder
• Deconvolution-Based Global Decoding for Neural Machine Translation
• Multi-Source Neural Machine Translation with Data Augmentation
• Improving the Performance of Neural Machine Translation Involving Morphologically Rich Languages
• Hybrid Self-Attention Network for Machine Translation
• Modeling Coherence for Discourse Neural Machine Translation
• SMT vs NMT: A Comparison over Hindi & Bengali Simple Sentences
• Augmenting Statistical Machine Translation with Subword Translation of Out-of-Vocabulary Words
• Neural Phrase-to-Phrase Machine Translation
• Improving Neural Machine Translation through Phrase-based Forced Decoding
• Toward a full-scale neural machine translation in production: the Booking.com use case
• Improving a Multi-Source Neural Machine Translation Model with Corpus Extension for Low-Resource Languages
• Modeling Latent Sentence Structure in Neural Machine Translation
• (Self-Attentive) Autoencoder-based Universal Language Representation for Machine Translation
• Modeling Recurrence for Transformer
• Differentiable lower bound for expected BLEU score
• Paying Attention to Multi-Word Expressions in Neural Machine Translation
• Self-Attentive Model for Headline Generation
• STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework
• Attention-based Vocabulary Selection for NMT Decoding
• Faster decoding for subword level Phrase-based SMT between related languages
• A Brief Survey of Multilingual Neural Machine Translation
• Variational Self-attention Model for Sentence Representation
• Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation
• Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory
• Learning to Translate in Real-time with Neural Machine Translation
• Debugging Neural Machine Translations
• Sentence Encoding with Tree-constrained Relation Networks
• Multi-channel Encoder for Neural Machine Translation
• The RGNLP Machine Translation Systems for WAT 2018
• Neural Machine Translation with Characters and Hierarchical Encoding