DCI-ES: An Extended Disentanglement Framework with Connections to Identifiability
arXiv:2210.00364
Abstract
In representation learning, a common approach is to seek representations which disentangle the underlying factors of variation. Eastwood & Williams (2018) proposed three metrics for quantifying the quality of such disentangled representations: disentanglement (D), completeness (C) and informativeness (I). In this work, we first connect this DCI framework to two common notions of linear and nonlinear identifiability, thereby establishing a formal link between disentanglement and the closely-related field of independent component analysis. We then propose an extended DCI-ES framework with two new measures of representation quality - explicitness (E) and size (S) - and point out how D and C can be computed for black-box predictors. Our main idea is that the functional capacity required to use a representation is an important but thus-far neglected aspect of representation quality, which we quantify using explicitness or ease-of-use (E). We illustrate the relevance of our extensions on the MPI3D and Cars3D datasets.
arXiv commentAccepted to ICLR 2023
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Auto-Encoding Variational Bayes
- Scaling Laws for Neural Language Models
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
- Deep Convolutional Inverse Graphics Network
- Disentangling by Factorising
- Understanding Global Feature Contributions With Additive Importance Measures
- Fantastic Generalization Measures and Where to Find Them
- Variational Autoencoders and Nonlinear ICA: A Unifying Framework
- Learning Deep Disentangled Embeddings with the F-Statistic Loss
- Weakly Supervised Disentanglement with Guarantees
- On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset
- A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation