Capturing Label Characteristics in VAEs
arXiv:2006.10102
Abstract
We present a principled approach to incorporating labels in VAEs that captures the rich characteristic information associated with those labels. While prior work has typically conflated these by learning latent variables that directly correspond to label values, we argue this is contrary to the intended effect of supervision in VAEs-capturing rich label characteristics with the latents. For example, we may want to capture the characteristics of a face that make it look young, rather than just the age of the person. To this end, we develop the CCVAE, a novel VAE model and concomitant variational objective which captures label characteristics explicitly in the latent space, eschewing direct correspondences between label values and latents. Through judicious structuring of mappings between such characteristic latents and labels, we show that the CCVAE can effectively learn meaningful representations of the characteristics of interest across a variety of supervision schemes. In particular, we show that the CCVAE allows for more effective and more general interventions to be performed, such as smooth traversals within the characteristics for a given label, diverse conditional generation, and transferring characteristics across datapoints.
arXiv commentAccepted to ICLR 2021
References in corpus (12)
- Auto-Encoding Variational Bayes
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Semi-Supervised Learning with Deep Generative Models
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Disentangling by Factorising
- Auxiliary Deep Generative Models
- Understanding Measures of Uncertainty for Adversarial Example Detection
- BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling
- Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models
- DIVA: Domain Invariant Variational Autoencoders
- DNA-GAN: Learning Disentangled Representations from Multi-Attribute Images
- Semi-Supervised Generation with Cluster-aware Generative Models