Bayesian Semisupervised Learning with Deep Generative Models

Abstract

Neural network based generative models with discriminative components are a powerful approach for semi-supervised learning. However, these techniques a) cannot account for model uncertainty in the estimation of the model's discriminative component and b) lack flexibility to capture complex stochastic patterns in the label generation process. To avoid these problems, we first propose to use a discriminative component with stochastic inputs for increased noise flexibility. We show how an efficient Gibbs sampling procedure can marginalize the stochastic inputs when inferring missing labels in this model. Following this, we extend the discriminative component to be fully Bayesian and produce estimates of uncertainty in its parameter values. This opens the door for semi-supervised Bayesian active learning.

References in corpus (8)

• Categorical Reparameterization with Gumbel-Softmax
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Weight Uncertainty in Neural Networks
• Deep Bayesian Active Learning with Image Data
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• Auxiliary Deep Generative Models
• Improving Variational Auto-Encoders using Householder Flow
• Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning

Cited by in corpus (6)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Incremental Deep Learning for Robust Object Detection in Unknown Cluttered Environments
• Featurized Density Ratio Estimation
• A Deep Generative Model for Semi-Supervised Classification with Noisy Labels
• Investigating the Effect of Intraclass Variability in Temporal Ensembling
• Semi-supervised learning objectives as log-likelihoods in a generative model of data curation