Evaluating the Disentanglement of Deep Generative Models through Manifold Topology

Abstract

Learning disentangled representations is regarded as a fundamental task for improving the generalization, robustness, and interpretability of generative models. However, measuring disentanglement has been challenging and inconsistent, often dependent on an ad-hoc external model or specific to a certain dataset. To address this, we present a method for quantifying disentanglement that only uses the generative model, by measuring the topological similarity of conditional submanifolds in the learned representation. This method showcases both unsupervised and supervised variants. To illustrate the effectiveness and applicability of our method, we empirically evaluate several state-of-the-art models across multiple datasets. We find that our method ranks models similarly to existing methods. We make ourcode publicly available at https://github.com/stanfordmlgroup/disentanglement.

arXiv commentPublished at ICLR 2021

References in corpus (17)

• Scikit-learn: Machine Learning in Python
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Disentangling by Factorising
• Statistical topological data analysis using persistence landscapes
• Learning with a Wasserstein Loss
• Towards a Definition of Disentangled Representations
• Weakly Supervised Disentanglement with Guarantees
• Geometry Score: A Method For Comparing Generative Adversarial Networks
• Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology
• On Efficient Optimal Transport: An Analysis of Greedy and Accelerated Mirror Descent Algorithms
• Bayesian representation learning with oracle constraints
• A Survey of Inductive Biases for Factorial Representation-Learning
• Unsupervised Model Selection for Variational Disentangled Representation Learning
• Vietoris-Rips Persistent Homology, Injective Metric Spaces, and The Filling Radius
• Disentangling Space and Time in Video with Hierarchical Variational Auto-encoders
• Wasserstein Barycenter Model Ensembling

Cited by in corpus (4)

• On Disentangled Representations Learned From Correlated Data
• Activation Landscapes as a Topological Summary of Neural Network Performance
• Measuring the Biases and Effectiveness of Content-Style Disentanglement
• Disentanglement Learning via Topology