Variational Neural Machine Translation

Abstract

Models of neural machine translation are often from a discriminative family of encoderdecoders that learn a conditional distribution of a target sentence given a source sentence. In this paper, we propose a variational model to learn this conditional distribution for neural machine translation: a variational encoderdecoder model that can be trained end-to-end. Different from the vanilla encoder-decoder model that generates target translations from hidden representations of source sentences alone, the variational model introduces a continuous latent variable to explicitly model underlying semantics of source sentences and to guide the generation of target translations. In order to perform efficient posterior inference and large-scale training, we build a neural posterior approximator conditioned on both the source and the target sides, and equip it with a reparameterization technique to estimate the variational lower bound. Experiments on both Chinese-English and English- German translation tasks show that the proposed variational neural machine translation achieves significant improvements over the vanilla neural machine translation baselines.

10 pages, accepted at emnlp 2016

References in corpus (2)

• Sequence to Sequence Learning with Neural Networks
• DRAW: A Recurrent Neural Network For Image Generation

Cited by in corpus (27)

• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Imagination improves Multimodal Translation
• A Survey of Deep Learning Techniques for Neural Machine Translation
• Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
• Topic-Guided Variational Autoencoders for Text Generation
• Analyzing Uncertainty in Neural Machine Translation
• Conditional Variational Autoencoder for Neural Machine Translation
• Spherical Latent Spaces for Stable Variational Autoencoders
• Variational Knowledge Graph Reasoning
• Variational Attention for Sequence-to-Sequence Models
• Unveiling the frontiers of deep learning: innovations shaping diverse domains
• Self-Attentive Residual Decoder for Neural Machine Translation
• Dirichlet Variational Autoencoder for Text Modeling
• Deep Neural Machine Translation with Linear Associative Unit
• Sequence to Sequence Mixture Model for Diverse Machine Translation
• Variational Recurrent Neural Machine Translation
• Natural Language Generation with Neural Variational Models
• Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction
• Bottleneck Conditional Density Estimation
• Large-scale Pretraining for Neural Machine Translation with Tens of Billions of Sentence Pairs
• (Self-Attentive) Autoencoder-based Universal Language Representation for Machine Translation
• Recurrent Neural Network-Based Semantic Variational Autoencoder for Sequence-to-Sequence Learning
• Hierarchical CVAE for Fine-Grained Hate Speech Classification
• Towards Interlingua Neural Machine Translation
• Bayesian Attention Modules
• Discrete Auto-regressive Variational Attention Models for Text Modeling
• Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder