Relevance Factor VAE: Learning and Identifying Disentangled Factors

Abstract

We propose a novel VAE-based deep auto-encoder model that can learn disentangled latent representations in a fully unsupervised manner, endowed with the ability to identify all meaningful sources of variation and their cardinality. Our model, dubbed Relevance-Factor-VAE, leverages the total correlation (TC) in the latent space to achieve the disentanglement goal, but also addresses the key issue of existing approaches which cannot distinguish between meaningful and nuisance factors of latent variation, often the source of considerable degradation in disentanglement performance. We tackle this issue by introducing the so-called relevance indicator variables that can be automatically learned from data, together with the VAE parameters. Our model effectively focuses the TC loss onto the relevant factors only by tolerating large prior KL divergences, a desideratum justified by our semi-parametric theoretical analysis. Using a suite of disentanglement metrics, including a newly proposed one, as well as qualitative evidence, we demonstrate that our model outperforms existing methods across several challenging benchmark datasets.

References in corpus (10)

• Auto-Encoding Variational Bayes
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Deep Convolutional Inverse Graphics Network
• Disentangling by Factorising
• Estimating divergence functionals and the likelihood ratio by convex risk minimization
• Adversarial Autoencoders
• Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
• Learning Disentangled Joint Continuous and Discrete Representations
• Learning Independent Features with Adversarial Nets for Non-linear ICA
• Understanding Visual Concepts with Continuation Learning

Cited by in corpus (5)

• ControlVAE: Controllable Variational Autoencoder
• Benchmarks, Algorithms, and Metrics for Hierarchical Disentanglement
• Tuning-Free Disentanglement via Projection
• EditVAE: Unsupervised Part-Aware Controllable 3D Point Cloud Shape Generation
• Bayes-Factor-VAE: Hierarchical Bayesian Deep Auto-Encoder Models for Factor Disentanglement