Auxiliary Deep Generative Models
arXiv:1602.05473
Abstract
Deep generative models parameterized by neural networks have recently achieved state-of-the-art performance in unsupervised and semi-supervised learning. We extend deep generative models with auxiliary variables which improves the variational approximation. The auxiliary variables leave the generative model unchanged but make the variational distribution more expressive. Inspired by the structure of the auxiliary variable we also propose a model with two stochastic layers and skip connections. Our findings suggest that more expressive and properly specified deep generative models converge faster with better results. We show state-of-the-art performance within semi-supervised learning on MNIST, SVHN and NORB datasets.
arXiv commentProceedings of the 33rd International Conference on Machine Learning, New York, NY, USA, 2016, JMLR: Workshop and Conference Proceedings volume 48, Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA, 2016
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Pixel Recurrent Neural Networks
- Theano: new features and speed improvements
- Distributional Smoothing with Virtual Adversarial Training
- Ladder Variational Autoencoders
- Hierarchical Variational Models
Cited by in corpus (110)
- An Introduction to Variational Autoencoders
- Unsupervised Data Augmentation for Consistency Training
- Temporal Ensembling for Semi-Supervised Learning
- Density estimation using Real NVP
- Adversarially Learned Inference
- Unsupervised Image-to-Image Translation Networks
- NVAE: A Deep Hierarchical Variational Autoencoder
- Causal Effect Inference with Deep Latent-Variable Models
- Neural Photo Editing with Introspective Adversarial Networks
- Ladder Variational Autoencoders
- An Overview of Deep Semi-Supervised Learning
- Self-training with Noisy Student improves ImageNet classification
- Good Semi-supervised Learning that Requires a Bad GAN
- Triple Generative Adversarial Nets
- Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
- Adversarial Training Methods for Semi-Supervised Text Classification
- Dynamical Variational Autoencoders: A Comprehensive Review
- Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks
- Towards a Neural Statistician
- Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities
- BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling
- Semi-supervised Learning with GANs: Manifold Invariance with Improved Inference
- Deep Active Inference
- Inference Suboptimality in Variational Autoencoders
- Semi-supervised Deep Generative Modelling of Incomplete Multi-Modality Emotional Data
- GANmapper: geographical data translation
- Semi-Implicit Graph Variational Auto-Encoders
- Uncertainty Estimations by Softplus normalization in Bayesian Convolutional Neural Networks with Variational Inference
- Operator Variational Inference
- Deep Neural Generative Model of Functional MRI Images for Psychiatric Disorder Diagnosis
- Learning by Association - A versatile semi-supervised training method for neural networks
- Adversarial Dropout Regularization
- Variational Sequential Monte Carlo
- Normalizing Flows on Riemannian Manifolds
- Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance
- Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks
- Tighter Variational Bounds are Not Necessarily Better
- GAR: An efficient and scalable Graph-based Activity Regularization for semi-supervised learning
- Structured Generative Adversarial Networks
- Semi-Implicit Variational Inference
- Learning Loss Functions for Semi-supervised Learning via Discriminative Adversarial Networks
- Multiple Causal Inference with Latent Confounding
- Self-Supervised Variational Auto-Encoders
- Dual Student: Breaking the Limits of the Teacher in Semi-supervised Learning
- Gaussian Process Prior Variational Autoencoders
- Hierarchical Variational Models
- Hierarchical Adversarially Learned Inference
- Bayesian Semisupervised Learning with Deep Generative Models
- A Review of Learning with Deep Generative Models from Perspective of Graphical Modeling
- A Probabilistic Framework for Deep Learning
- EnAET: A Self-Trained framework for Semi-Supervised and Supervised Learning with Ensemble Transformations
- Reparameterization Gradients through Acceptance-Rejection Sampling Algorithms
- Bringing Giant Neural Networks Down to Earth with Unlabeled Data
- Hidden Talents of the Variational Autoencoder
- Disentangled Variational Auto-Encoder for Semi-supervised Learning
- Variational Laplace Autoencoders
- Semi-Supervised Generation with Cluster-aware Generative Models
- Semi-Supervised Learning using Siamese Networks
- Variational f-divergence Minimization
- Leveraging the Exact Likelihood of Deep Latent Variable Models
- Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning
- Triple Generative Adversarial Networks
- Weakly Supervised Clustering by Exploiting Unique Class Count
- Hierarchical VAEs Know What They Don't Know
- Information Losses in Neural Classifiers from Sampling
- Bottleneck Conditional Density Estimation
- Classification-Reconstruction Learning for Open-Set Recognition
- Variational Inference for Computational Imaging Inverse Problems
- Unsupervised Multi-Domain Image Translation with Domain-Specific Encoders/Decoders
- Capturing Label Characteristics in VAEs
- Generate High Resolution Images With Generative Variational Autoencoder
- Multi-Task Variational Information Bottleneck
- Manifold Adversarial Learning
- Semi-Supervised Learning with the Deep Rendering Mixture Model
- A Novel Variational Autoencoder with Applications to Generative Modelling, Classification, and Ordinal Regression
- Trust-Region Variational Inference with Gaussian Mixture Models
- Flow Contrastive Estimation of Energy-Based Models
- Semi-Supervised Learning Enabled by Multiscale Deep Neural Network Inversion
- Direct Evolutionary Optimization of Variational Autoencoders With Binary Latents
- Auto-clustering Output Layer: Automatic Learning of Latent Annotations in Neural Networks
- Learning Sparse, Distributed Representations using the Hebbian Principle
- Semi-supervised Disentanglement with Independent Vector Variational Autoencoders
- Variational Inference with Holder Bounds
- Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Models
- On the Transformation of Latent Space in Autoencoders
- Semi-Supervised Learning via New Deep Network Inversion
- Doubly Semi-Implicit Variational Inference
- Reliable Semi-Supervised Learning when Labels are Missing at Random
- General Control Functions for Causal Effect Estimation from Instrumental Variables
- Amortized Population Gibbs Samplers with Neural Sufficient Statistics
- Max-Margin Deep Generative Models for (Semi-)Supervised Learning
- Inverting Variational Autoencoders for Improved Generative Accuracy
- Rethinking conditional GAN training: An approach using geometrically structured latent manifolds
- Understanding and Improving Virtual Adversarial Training
- Local Clustering with Mean Teacher for Semi-supervised Learning
- Neural Variational Inference and Learning in Undirected Graphical Models
- Energy-Inspired Models: Learning with Sampler-Induced Distributions
- A Contemporary Overview of Probabilistic Latent Variable Models
- Excavate Condition-invariant Space by Intrinsic Encoder
- Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Sampling
- Data Augmentation for Deep Candlestick Learner
- Matching Embeddings for Domain Adaptation
- Semi-Supervised Self-Growing Generative Adversarial Networks for Image Recognition
- Learning Consistent Deep Generative Models from Sparse Data via Prediction Constraints
- Probabilistic Autoencoder using Fisher Information
- Deep Dive into Semi-Supervised ELBO for Improving Classification Performance
- EXoN: EXplainable encoder Network
- Learning Deep-Latent Hierarchies by Stacking Wasserstein Autoencoders
- Thompson Sampling via Local Uncertainty
- From abstract items to latent spaces to observed data and back: Compositional Variational Auto-Encoder