BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling

Abstract

With the introduction of the variational autoencoder (VAE), probabilistic latent variable models have received renewed attention as powerful generative models. However, their performance in terms of test likelihood and quality of generated samples has been surpassed by autoregressive models without stochastic units. Furthermore, flow-based models have recently been shown to be an attractive alternative that scales well to high-dimensional data. In this paper we close the performance gap by constructing VAE models that can effectively utilize a deep hierarchy of stochastic variables and model complex covariance structures. We introduce the Bidirectional-Inference Variational Autoencoder (BIVA), characterized by a skip-connected generative model and an inference network formed by a bidirectional stochastic inference path. We show that BIVA reaches state-of-the-art test likelihoods, generates sharp and coherent natural images, and uses the hierarchy of latent variables to capture different aspects of the data distribution. We observe that BIVA, in contrast to recent results, can be used for anomaly detection. We attribute this to the hierarchy of latent variables which is able to extract high-level semantic features. Finally, we extend BIVA to semi-supervised classification tasks and show that it performs comparably to state-of-the-art results by generative adversarial networks.

References in corpus (26)

• Adam: A Method for Stochastic Optimization
• WaveNet: A Generative Model for Raw Audio
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• DRAW: A Recurrent Neural Network For Image Generation
• Density estimation using Real NVP
• Semi-Supervised Learning with Ladder Networks
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Distributional Smoothing with Virtual Adversarial Training
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
• Auxiliary Deep Generative Models
• Variational Lossy Autoencoder
• Triple Generative Adversarial Nets
• Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
• Neural Variational Inference and Learning in Belief Networks
• Towards Deeper Understanding of Variational Autoencoding Models
• Variational Bayesian Inference with Stochastic Search
• Improving Variational Auto-Encoders using Householder Flow
• A Hybrid Convolutional Variational Autoencoder for Text Generation
• DVAE++: Discrete Variational Autoencoders with Overlapping Transformations
• Avoiding Latent Variable Collapse With Generative Skip Models
• The Variational Gaussian Process
• Semi-Supervised Generation with Cluster-aware Generative Models

Cited by in corpus (44)

• Denoising Diffusion Probabilistic Models
• NVAE: A Deep Hierarchical Variational Autoencoder
• Variational Autoencoders and Nonlinear ICA: A Unifying Framework
• DIVA: Domain Invariant Variational Autoencoders
• Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model
• IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
• Simple and Effective VAE Training with Calibrated Decoders
• Automated curricula through setter-solver interactions
• Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Features
• On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition
• The Usual Suspects? Reassessing Blame for VAE Posterior Collapse
• A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities
• Hierarchical Quantized Autoencoders
• Self-Supervised Variational Auto-Encoders
• Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse
• Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
• Score-based Generative Modeling in Latent Space
• VQ-GNN: A Universal Framework to Scale up Graph Neural Networks using Vector Quantization
• Further Analysis of Outlier Detection with Deep Generative Models
• On the Necessity and Effectiveness of Learning the Prior of Variational Auto-Encoder
• HiLLoC: Lossless Image Compression with Hierarchical Latent Variable Models
• A Contrastive Learning Approach for Training Variational Autoencoder Priors
• Innovations Autoencoder and its Application in One-class Anomalous Sequence Detection
• Hierarchical VAEs Know What They Don't Know
• Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction
• Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory
• Capturing Label Characteristics in VAEs
• Improving VAEs' Robustness to Adversarial Attack
• SCHA-VAE: Hierarchical Context Aggregation for Few-Shot Generation
• Generative Hierarchical Models for Parts, Objects, and Scenes
• Reducing the Computational Cost of Deep Generative Models with Binary Neural Networks
• Bigeminal Priors Variational auto-encoder
• Optimal Variance Control of the Score Function Gradient Estimator for Importance Weighted Bounds
• Diagnosing Vulnerability of Variational Auto-Encoders to Adversarial Attacks
• A Quaternion-Valued Variational Autoencoder
• Conditional Generative Modeling via Learning the Latent Space
• Relaxed-Responsibility Hierarchical Discrete VAEs
• OMASGAN: Out-of-Distribution Minimum Anomaly Score GAN for Sample Generation on the Boundary
• Learning Deep-Latent Hierarchies by Stacking Wasserstein Autoencoders
• Context-encoding Variational Autoencoder for Unsupervised Anomaly Detection -- Short Paper
• Estimating Disentangled Belief about Hidden State and Hidden Task for Meta-RL
• Boxhead: A Dataset for Learning Hierarchical Representations