VAE with a VampPrior

Abstract

Many different methods to train deep generative models have been introduced in the past. In this paper, we propose to extend the variational auto-encoder (VAE) framework with a new type of prior which we call "Variational Mixture of Posteriors" prior, or VampPrior for short. The VampPrior consists of a mixture distribution (e.g., a mixture of Gaussians) with components given by variational posteriors conditioned on learnable pseudo-inputs. We further extend this prior to a two layer hierarchical model and show that this architecture with a coupled prior and posterior, learns significantly better models. The model also avoids the usual local optima issues related to useless latent dimensions that plague VAEs. We provide empirical studies on six datasets, namely, static and binary MNIST, OMNIGLOT, Caltech 101 Silhouettes, Frey Faces and Histopathology patches, and show that applying the hierarchical VampPrior delivers state-of-the-art results on all datasets in the unsupervised permutation invariant setting and the best results or comparable to SOTA methods for the approach with convolutional networks.

16 pages, final version, AISTATS 2018

References in corpus (5)

• NICE: Non-linear Independent Components Estimation
• Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Variational Lossy Autoencoder
• Fixing a Broken ELBO

Cited by in corpus (45)

• State Predictive Information Bottleneck
• Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis
• Fixing a Broken ELBO
• DVAE++: Discrete Variational Autoencoders with Overlapping Transformations
• Avoiding Latent Variable Collapse With Generative Skip Models
• Diffusion Variational Autoencoders
• Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models
• Poincaré Wasserstein Autoencoder
• Self-Supervised Variational Auto-Encoders
• Gaussian Process Prior Variational Autoencoders
• Generative Latent Flow
• Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse
• Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection
• Degeneration in VAE: in the Light of Fisher Information Loss
• PixelVAE++: Improved PixelVAE with Discrete Prior
• Associative Compression Networks for Representation Learning
• Preventing Posterior Collapse with delta-VAEs
• VAEM: a Deep Generative Model for Heterogeneous Mixed Type Data
• Nonparametric Inference for Auto-Encoding Variational Bayes
• InfoCatVAE: Representation Learning with Categorical Variational Autoencoders
• Variational Autoencoders with Riemannian Brownian Motion Priors
• Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations
• Variational Variance: Simple, Reliable, Calibrated Heteroscedastic Noise Variance Parameterization
• Sampling-based probabilistic inference emerges from learning in neural circuits with a cost on reliability
• Jigsaw-VAE: Towards Balancing Features in Variational Autoencoders
• Anomaly scores for generative models
• Manifolds for Unsupervised Visual Anomaly Detection
• Variational Diffusion Autoencoders with Random Walk Sampling
• Hypernetwork approach to generating point clouds
• Improved Variational Neural Machine Translation by Promoting Mutual Information
• Dueling Decoders: Regularizing Variational Autoencoder Latent Spaces
• Doubly Semi-Implicit Variational Inference
• MIM: Mutual Information Machine
• Sparse encoding for more-interpretable feature-selecting representations in probabilistic matrix factorization
• NeoNav: Improving the Generalization of Visual Navigation via Generating Next Expected Observations
• Benefiting Deep Latent Variable Models via Learning the Prior and Removing Latent Regularization
• Bridging the ELBO and MMD
• MaskAAE: Latent space optimization for Adversarial Auto-Encoders
• Characterizing and Avoiding Problematic Global Optima of Variational Autoencoders
• D2C: Diffusion-Denoising Models for Few-shot Conditional Generation
• Learning Priors for Adversarial Autoencoders
• Decoupled Variational Embedding for Signed Directed Networks
• $\rho$-VAE: Autoregressive parametrization of the VAE encoder
• High Mutual Information in Representation Learning with Symmetric Variational Inference
• Concept Formation and Dynamics of Repeated Inference in Deep Generative Models