Disentanglement Learning via Topology

Abstract

We propose TopDis (Topological Disentanglement), a method for learning disentangled representations via adding a multi-scale topological loss term. Disentanglement is a crucial property of data representations substantial for the explainability and robustness of deep learning models and a step towards high-level cognition. The state-of-the-art methods are based on VAE and encourage the joint distribution of latent variables to be factorized. We take a different perspective on disentanglement by analyzing topological properties of data manifolds. In particular, we optimize the topological similarity for data manifolds traversals. To the best of our knowledge, our paper is the first one to propose a differentiable topological loss for disentanglement learning. Our experiments have shown that the proposed TopDis loss improves disentanglement scores such as MIG, FactorVAE score, SAP score, and DCI disentanglement score with respect to state-of-the-art results while preserving the reconstruction quality. Our method works in an unsupervised manner, permitting us to apply it to problems without labeled factors of variation. The TopDis loss works even when factors of variation are correlated. Additionally, we show how to use the proposed topological loss to find disentangled directions in a trained GAN.

References in corpus (20)

• Auto-Encoding Variational Bayes
• Semi-Supervised Learning with Deep Generative Models
• Deep Convolutional Inverse Graphics Network
• Disentangling by Factorising
• Towards a Definition of Disentangled Representations
• Groupoids: unifying internal and external symmetry
• Unsupervised Learning of Disentangled Representations from Video
• Learning Disentangled Representations with Semi-Supervised Deep Generative Models
• Learning Deep Disentangled Embeddings with the F-Statistic Loss
• Disentangling Factors of Variation via Generative Entangling
• On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset
• Learning the Irreducible Representations of Commutative Lie Groups
• Regularizing Deep Multi-Task Networks using Orthogonal Gradients
• A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation
• The Geometry of Deep Generative Image Models and its Applications
• Dual Swap Disentangling
• Evaluating the Disentanglement of Deep Generative Models through Manifold Topology
• Disentanglement of Correlated Factors via Hausdorff Factorized Support
• Learning Topology-Preserving Data Representations
• Towards Building A Group-based Unsupervised Representation Disentanglement Framework