Neural Machine Translation: A Review and Survey

Abstract

The field of machine translation (MT), the automatic translation of written text from one natural language into another, has experienced a major paradigm shift in recent years. Statistical MT, which mainly relies on various count-based models and which used to dominate MT research for decades, has largely been superseded by neural machine translation (NMT), which tackles translation with a single neural network. In this work we will trace back the origins of modern NMT architectures to word and sentence embeddings and earlier examples of the encoder-decoder network family. We will conclude with a survey of recent trends in the field.

Extended version of "Neural Machine Translation: A Review" accepted by the Journal of Artificial Intelligence Research (JAIR)

References in corpus (79)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Distilling the Knowledge in a Neural Network
• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• ADADELTA: An Adaptive Learning Rate Method
• Natural Language Processing (almost) from Scratch
• Neural Architecture Search with Reinforcement Learning
• On the difficulty of training Recurrent Neural Networks
• Towards A Rigorous Science of Interpretable Machine Learning
• Methods for Interpreting and Understanding Deep Neural Networks
• A Structured Self-attentive Sentence Embedding
• Theano: new features and speed improvements
• Recurrent Models of Visual Attention
• Multiple Object Recognition with Visual Attention
• To prune, or not to prune: exploring the efficacy of pruning for model compression
• Dual Learning for Machine Translation
• On Using Monolingual Corpora in Neural Machine Translation
• Attention is not Explanation
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• Regularizing Neural Networks by Penalizing Confident Output Distributions
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Pay Less Attention with Lightweight and Dynamic Convolutions
• Neural Machine Translation in Linear Time
• Assessing BERT's Syntactic Abilities
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• Depthwise Separable Convolutions for Neural Machine Translation
• Better Mixing via Deep Representations
• The Evolved Transformer
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
• Fast Domain Adaptation for Neural Machine Translation
• Insertion Transformer: Flexible Sequence Generation via Insertion Operations
• An Empirical Model of Large-Batch Training
• Weighted Transformer Network for Machine Translation
• Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
• Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation
• THUMT: An Open Source Toolkit for Neural Machine Translation
• Adding Interpretable Attention to Neural Translation Models Improves Word Alignment
• SYSTRAN's Pure Neural Machine Translation Systems
• On the Computational Efficiency of Training Neural Networks
• No Training Required: Exploring Random Encoders for Sentence Classification
• Distance-based Self-Attention Network for Natural Language Inference
• Pre-Translation for Neural Machine Translation
• Notes on Noise Contrastive Estimation and Negative Sampling
• Lattice-Based Recurrent Neural Network Encoders for Neural Machine Translation
• RNN Approaches to Text Normalization: A Challenge
• Bridging Neural Machine Translation and Bilingual Dictionaries
• Language Models with Transformers
• Effective Strategies in Zero-Shot Neural Machine Translation
• Calibration of Encoder Decoder Models for Neural Machine Translation
• Vocabulary Selection Strategies for Neural Machine Translation
• End-to-End Speech Translation with Knowledge Distillation
• Dynamic Evaluation of Transformer Language Models
• Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder
• Neural machine translation for low-resource languages
• Assessing the Tolerance of Neural Machine Translation Systems Against Speech Recognition Errors
• Context in Neural Machine Translation: A Review of Models and Evaluations
• NoiseOut: A Simple Way to Prune Neural Networks
• Neutron: An Implementation of the Transformer Translation Model and its Variants
• QCRI Machine Translation Systems for IWSLT 16
• The Neural Noisy Channel
• Confidence through Attention
• Unsupervised Neural Machine Translation with SMT as Posterior Regularization
• Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input
• Competence-based Curriculum Learning for Neural Machine Translation
• A Fully Differentiable Beam Search Decoder
• Improving the Performance of Neural Machine Translation Involving Morphologically Rich Languages
• SMT vs NMT: A Comparison over Hindi & Bengali Simple Sentences
• Improving Neural Machine Translation through Phrase-based Forced Decoding
• Toward a full-scale neural machine translation in production: the Booking.com use case
• Modeling Recurrence for Transformer
• Self-Attentive Model for Headline Generation
• Attention-based Vocabulary Selection for NMT Decoding
• Faster decoding for subword level Phrase-based SMT between related languages
• Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation
• Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory
• A Brief Survey of Multilingual Neural Machine Translation
• Learning to Translate in Real-time with Neural Machine Translation
• Multi-channel Encoder for Neural Machine Translation
• Neural Machine Translation with Characters and Hierarchical Encoding