Improving Variational Auto-Encoders using Householder Flow
arXiv:1611.09630
Abstract
Variational auto-encoders (VAE) are scalable and powerful generative models. However, the choice of the variational posterior determines tractability and flexibility of the VAE. Commonly, latent variables are modeled using the normal distribution with a diagonal covariance matrix. This results in computational efficiency but typically it is not flexible enough to match the true posterior distribution. One fashion of enriching the variational posterior distribution is application of normalizing flows, i.e., a series of invertible transformations to latent variables with a simple posterior. In this paper, we follow this line of thinking and propose a volume-preserving flow that uses a series of Householder transformations. We show empirically on MNIST dataset and histopathology data that the proposed flow allows to obtain more flexible variational posterior and competitive results comparing to other normalizing flows.
A corrected version of the paper submitted to Bayesian Deep Learning Workshop (NIPS 2016)
References in corpus (2)
Cited by in corpus (27)
- Dual Adversarial Auto-Encoders for Clustering
- Topic-Guided Variational Autoencoders for Text Generation
- Trivializations for Gradient-Based Optimization on Manifolds
- Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models
- Learning Symmetries of Classical Integrable Systems
- Bayesian Semisupervised Learning with Deep Generative Models
- iUNets: Fully invertible U-Nets with Learnable Up- and Downsampling
- Deep Learning Models for Digital Pathology
- Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling
- Multimodal Generative Models for Compositional Representation Learning
- Recursive Inference for Variational Autoencoders
- Max-Affine Spline Insights into Deep Generative Networks
- What if Neural Networks had SVDs?
- Learning normalizing flows from Entropy-Kantorovich potentials
- Reducing the Amortization Gap in Variational Autoencoders: A Bayesian Random Function Approach
- Sum-Product-Transform Networks: Exploiting Symmetries using Invertible Transformations
- D2C: Diffusion-Denoising Models for Few-shot Conditional Generation
- Information Theoretic Lower Bounds on Negative Log Likelihood
- One Reflection Suffice
- LaDDer: Latent Data Distribution Modelling with a Generative Prior
- Ordering Dimensions with Nested Dropout Normalizing Flows
- Cauchy-Schwarz Regularized Autoencoder
- Training Invertible Linear Layers through Rank-One Perturbations
- ByPE-VAE: Bayesian Pseudocoresets Exemplar VAE
- Enhanced Variational Inference with Dyadic Transformation
- Viscos Flows: Variational Schur Conditional Sampling With Normalizing Flows
- Out-of-Distribution Detection of Melanoma using Normalizing Flows