Isolating Sources of Disentanglement in Variational Autoencoders

Abstract

We decompose the evidence lower bound to show the existence of a term measuring the total correlation between latent variables. We use this to motivate our $\beta$-TCVAE (Total Correlation Variational Autoencoder), a refinement of the state-of-the-art $\beta$-VAE objective for learning disentangled representations, requiring no additional hyperparameters during training. We further propose a principled classifier-free measure of disentanglement called the mutual information gap (MIG). We perform extensive quantitative and qualitative experiments, in both restricted and non-restricted settings, and show a strong relation between total correlation and disentanglement, when the latent variables model is trained using our framework.

References in corpus (18)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Deep Convolutional Inverse Graphics Network
• Disentangling by Factorising
• MINE: Mutual Information Neural Estimation
• InfoVAE: Information Maximizing Variational Autoencoders
• Deep Variational Information Bottleneck
• Amortised MAP Inference for Image Super-resolution
• Learning Disentangled Representations with Semi-Supervised Deep Generative Models
• Discovering Hidden Factors of Variation in Deep Networks
• Disentangling Factors of Variation via Generative Entangling
• Bayesian representation learning with oracle constraints
• Independently Controllable Factors
• Fixing a Broken ELBO
• A Survey of Inductive Biases for Factorial Representation-Learning
• Auto-Encoding Total Correlation Explanation
• Disentangling Space and Time in Video with Hierarchical Variational Auto-encoders
• Controllable Semantic Image Inpainting

Cited by in corpus (25)

• Causal Confusion in Imitation Learning
• Learning Disentangled Joint Continuous and Discrete Representations
• Disentangling Disentanglement in Variational Autoencoders
• IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks
• Contrastive Variational Autoencoder Enhances Salient Features
• ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA
• Targeted Neural Dynamical Modeling
• Morpho-MNIST: Quantitative Assessment and Diagnostics for Representation Learning
• Dual Swap Disentangling
• Learning and controlling the source-filter representation of speech with a variational autoencoder
• Unsupervised Learning of Compositional Energy Concepts
• Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity
• InfoCatVAE: Representation Learning with Categorical Variational Autoencoders
• Disentangling Controllable and Uncontrollable Factors of Variation by Interacting with the World
• Measuring non-trivial compositionality in emergent communication
• Diffusion Priors In Variational Autoencoders
• Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations
• Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems
• Unsupervised and interpretable scene discovery with Discrete-Attend-Infer-Repeat
• Practical and Consistent Estimation of f-Divergences
• Multimodal Representation Learning via Maximization of Local Mutual Information
• ByPE-VAE: Bayesian Pseudocoresets Exemplar VAE
• Revisiting Factorizing Aggregated Posterior in Learning Disentangled Representations
• A Factorial Mixture Prior for Compositional Deep Generative Models
• Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation