Diagnosing and Enhancing VAE Models

Abstract

Although variational autoencoders (VAEs) represent a widely influential deep generative model, many aspects of the underlying energy function remain poorly understood. In particular, it is commonly believed that Gaussian encoder/decoder assumptions reduce the effectiveness of VAEs in generating realistic samples. In this regard, we rigorously analyze the VAE objective, differentiating situations where this belief is and is not actually true. We then leverage the corresponding insights to develop a simple VAE enhancement that requires no additional hyperparameters or sensitive tuning. Quantitatively, this proposal produces crisp samples and stable FID scores that are actually competitive with a variety of GAN models, all while retaining desirable attributes of the original VAE architecture. A shorter version of this work will appear in the ICLR 2019 conference proceedings (Dai and Wipf, 2019). The code for our model is available at https://github.com/daib13/ TwoStageVAE.

References in corpus (16)

• Auto-Encoding Variational Bayes
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Neural Discrete Representation Learning
• Conditional Image Generation with PixelCNN Decoders
• Tutorial on Variational Autoencoders
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Autoencoding beyond pixels using a learned similarity metric
• Variational Lossy Autoencoder
• Importance Weighted Autoencoders
• On Convergence and Stability of GANs
• Demystifying MMD GANs
• Hyperspherical Variational Auto-Encoders
• Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
• From optimal transport to generative modeling: the VEGAN cookbook

Cited by in corpus (44)

• From Variational to Deterministic Autoencoders
• DIVA: Domain Invariant Variational Autoencoders
• Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• ControlVAE: Controllable Variational Autoencoder
• AstroVaDEr: Astronomical Variational Deep Embedder for Unsupervised Morphological Classification of Galaxies and Synthetic Image Generation
• Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images
• Generative Capacity of Probabilistic Protein Sequence Models
• Differentiable Strong Lensing: Uniting Gravity and Neural Nets through Differentiable Probabilistic Programming
• Simple and Effective VAE Training with Calibrated Decoders
• Anomaly localization by modeling perceptual features
• Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning
• Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse
• Generative Latent Flow
• VAEM: a Deep Generative Model for Heterogeneous Mixed Type Data
• Low-rank Characteristic Tensor Density Estimation Part II: Compression and Latent Density Estimation
• Regularized Autoencoders via Relaxed Injective Probability Flow
• Jacobian Adversarially Regularized Networks for Robustness
• Jigsaw-VAE: Towards Balancing Features in Variational Autoencoders
• Anomaly scores for generative models
• Learning to Synthesize Fashion Textures
• Re-parameterizing VAEs for stability
• TrustMAE: A Noise-Resilient Defect Classification Framework using Memory-Augmented Auto-Encoders with Trust Regions
• Exponential Tilting of Generative Models: Improving Sample Quality by Training and Sampling from Latent Energy
• Man-in-the-Middle Attacks against Machine Learning Classifiers via Malicious Generative Models
• Batch norm with entropic regularization turns deterministic autoencoders into generative models
• Achieving Explainability for Plant Disease Classification with Disentangled Variational Autoencoders
• MCEN: Bridging Cross-Modal Gap between Cooking Recipes and Dish Images with Latent Variable Model
• ControlVAE: Tuning, Analytical Properties, and Performance Analysis
• Variation Network: Learning High-level Attributes for Controlled Input Manipulation
• LDC-VAE: A Latent Distribution Consistency Approach to Variational AutoEncoders
• D2C: Diffusion-Denoising Models for Few-shot Conditional Generation
• Learning and Inference in Imaginary Noise Models
• Generalizing Variational Autoencoders with Hierarchical Empirical Bayes
• No Representation without Transformation
• Disentangled Neural Architecture Search
• A Generative Approach Towards Improved Robotic Detection of Marine Litter
• SynFace: Face Recognition with Synthetic Data
• Wavelets to the Rescue: Improving Sample Quality of Latent Variable Deep Generative Models
• $\rho$-VAE: Autoregressive parametrization of the VAE encoder
• Neighbor Embedding Variational Autoencoder
• Momentum Contrastive Autoencoder: Using Contrastive Learning for Latent Space Distribution Matching in WAE
• The Effects of Invertibility on the Representational Complexity of Encoders in Variational Autoencoders
• There and Back Again: Unraveling the Variational Auto-Encoder