Theory and Evaluation Metrics for Learning Disentangled Representations

Abstract

We make two theoretical contributions to disentanglement learning by (a) defining precise semantics of disentangled representations, and (b) establishing robust metrics for evaluation. First, we characterize the concept "disentangled representations" used in supervised and unsupervised methods along three dimensions-informativeness, separability and interpretability - which can be expressed and quantified explicitly using information-theoretic constructs. This helps explain the behaviors of several well-known disentanglement learning models. We then propose robust metrics for measuring informativeness, separability and interpretability. Through a comprehensive suite of experiments, we show that our metrics correctly characterize the representations learned by different methods and are consistent with qualitative (visual) results. Thus, the metrics allow disentanglement learning methods to be compared on a fair ground. We also empirically uncovered new interesting properties of VAE-based methods and interpreted them with our formulation. These findings are promising and hopefully will encourage the design of more theoretically driven models for learning disentangled representations.

References in corpus (8)

• Deep Convolutional Inverse Graphics Network
• Disentangling by Factorising
• Recent Advances in Autoencoder-Based Representation Learning
• Towards a Definition of Disentangled Representations
• Disentangling factors of variation in deep representations using adversarial training
• Learning Deep Disentangled Embeddings with the F-Statistic Loss
• SCAN: Learning Hierarchical Compositional Visual Concepts
• A Survey of Inductive Biases for Factorial Representation-Learning

Cited by in corpus (9)

• Commutative Lie Group VAE for Disentanglement Learning
• DynamicVAE: Decoupling Reconstruction Error and Disentangled Representation Learning
• Measuring the Biases and Effectiveness of Content-Style Disentanglement
• ControlVAE: Tuning, Analytical Properties, and Performance Analysis
• Semi-Disentangled Representation Learning in Recommendation System
• Evaluation of Latent Space Disentanglement in the Presence of Interdependent Attributes
• DEFT: Distilling Entangled Factors by Preventing Information Diffusion
• Disentangling Action Sequences: Discovering Correlated Samples
• Towards Better Understanding of Disentangled Representations via Mutual Information