An Identifiable Double VAE For Disentangled Representations
arXiv:2010.09360
Abstract
A large part of the literature on learning disentangled representations focuses on variational autoencoders (VAE). Recent developments demonstrate that disentanglement cannot be obtained in a fully unsupervised setting without inductive biases on models and data. However, Khemakhem et al., AISTATS, 2020 suggest that employing a particular form of factorized prior, conditionally dependent on auxiliary variables complementing input observations, can be one such bias, resulting in an identifiable model with guarantees on disentanglement. Working along this line, we propose a novel VAE-based generative model with theoretical guarantees on identifiability. We obtain our conditional prior over the latents by learning an optimal representation, which imposes an additional strength on their regularization. We also extend our method to semi-supervised settings. Experimental results indicate superior performance with respect to state-of-the-art approaches, according to several established metrics proposed in the literature on disentanglement.
References in corpus (16)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Auto-Encoding Variational Bayes
- The information bottleneck method
- Semi-Supervised Learning with Deep Generative Models
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Disentangling by Factorising
- Understanding disentangling in -VAE
- Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
- Discovering Hidden Factors of Variation in Deep Networks
- Learning Deep Disentangled Embeddings with the F-Statistic Loss
- Learning Latent Subspaces in Variational Autoencoders
- Weakly Supervised Disentanglement with Guarantees
- Disentangling Factors of Variation Using Few Labels
- PixelGAN Autoencoders
- On Implicit Regularization in -VAEs
- Robust Ordinal VAE: Employing Noisy Pairwise Comparisons for Disentanglement