Semi-Amortized Variational Autoencoders

Abstract

Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network. While AVI has enabled efficient training of deep generative models such as variational autoencoders (VAE), recent empirical work suggests that inference networks can produce suboptimal variational parameters. We propose a hybrid approach, to use AVI to initialize the variational parameters and run stochastic variational inference (SVI) to refine them. Crucially, the local SVI procedure is itself differentiable, so the inference network and generative model can be trained end-to-end with gradient-based optimization. This semi-amortized approach enables the use of rich generative models without experiencing the posterior-collapse phenomenon common in training VAEs for problems like text generation. Experiments show this approach outperforms strong autoregressive and variational baselines on standard text and image datasets.

arXiv commentICML 2018

References in corpus (26)

• Auto-Encoding Variational Bayes
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Conditional Image Generation with PixelCNN Decoders
• DRAW: A Recurrent Neural Network For Image Generation
• Unrolled Generative Adversarial Networks
• Gradient-based Hyperparameter Optimization through Reversible Learning
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Variational Lossy Autoencoder
• Importance Weighted Autoencoders
• Composing graphical models with neural networks for structured representations and fast inference
• Toward Controlled Generation of Text
• OptNet: Differentiable Optimization as a Layer in Neural Networks
• TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency
• Towards Deeper Understanding of Variational Autoencoding Models
• Towards Conceptual Compression
• Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
• Inference Suboptimality in Variational Autoencoders
• Topic Compositional Neural Language Model
• A Hybrid Convolutional Variational Autoencoder for Text Generation
• Nonparametric Variational Auto-encoders for Hierarchical Representation Learning
• Generating Sentences by Editing Prototypes
• Iterative Amortized Inference
• Z-Forcing: Training Stochastic Recurrent Networks
• On the challenges of learning with inference networks on sparse, high-dimensional data
• Advances in Variational Inference
• Importance Sampled Stochastic Optimization for Variational Inference

Cited by in corpus (21)

• Topic-Guided Variational Autoencoders for Text Generation
• Avoiding Latent Variable Collapse With Generative Skip Models
• Educating Text Autoencoders: Latent Representation Guidance via Denoising
• Action and Perception as Divergence Minimization
• Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse
• Structured Variational Inference for Simulating Populations of Radio Galaxies
• DLow: Diversifying Latent Flows for Diverse Human Motion Prediction
• MAE: Mutual Posterior-Divergence Regularization for Variational AutoEncoders
• Adaptive Path-Integral Autoencoder: Representation Learning and Planning for Dynamical Systems
• Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling
• mu-Forcing: Training Variational Recurrent Autoencoders for Text Generation
• Control as Hybrid Inference
• Certifiably Robust Variational Autoencoders
• Iterative VAE as a predictive brain model for out-of-distribution generalization
• Pseudo-Encoded Stochastic Variational Inference
• Bridging the ELBO and MMD
• Characterizing and Avoiding Problematic Global Optima of Variational Autoencoders
• Semi-Implicit Stochastic Recurrent Neural Networks
• Mutual Information Constraints for Monte-Carlo Objectives
• Disentangling Latent Space for VAE by Label Relevant/Irrelevant Dimensions
• Inverse Graphics: Unsupervised Learning of 3D Shapes from Single Images