Bottleneck Conditional Density Estimation

Abstract

We introduce a new framework for training deep generative models for high-dimensional conditional density estimation. The Bottleneck Conditional Density Estimator (BCDE) is a variant of the conditional variational autoencoder (CVAE) that employs layer(s) of stochastic variables as the bottleneck between the input $x$ and target $y$, where both are high-dimensional. Crucially, we propose a new hybrid training method that blends the conditional generative model with a joint generative model. Hybrid blending is the key to effective training of the BCDE, which avoids overfitting and provides a novel mechanism for leveraging unlabeled data. We show that our hybrid training procedure enables models to achieve competitive results in the MNIST quadrant prediction task in the fully-supervised setting, and sets new benchmarks in the semi-supervised regime for MNIST, SVHN, and CelebA.

References in corpus (9)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Density estimation using Real NVP
• A note on the evaluation of generative models
• Auxiliary Deep Generative Models
• Attribute2Image: Conditional Image Generation from Visual Attributes
• Fast Nonparametric Conditional Density Estimation
• Variational Neural Machine Translation
• Hierarchical Variational Models

Cited by in corpus (5)

• Generative Neural Machine Translation
• Robust Locally-Linear Controllable Embedding
• Deep Variational Sufficient Dimensionality Reduction
• Inverting Variational Autoencoders for Improved Generative Accuracy
• Semi-Supervised Few-Shot Classification with Deep Invertible Hybrid Models