Disentanglement Learning via Topology

Abstract

We propose TopDis (Topological Disentanglement), a method for learning disentangled representations via adding a multi-scale topological loss term. Disentanglement is a crucial property of data representations substantial for the explainability and robustness of deep learning models and a step towards high-level cognition. The state-of-the-art methods are based on VAE and encourage the joint distribution of latent variables to be factorized. We take a different perspective on disentanglement by analyzing topological properties of data manifolds. In particular, we optimize the topological similarity for data manifolds traversals. To the best of our knowledge, our paper is the first one to propose a differentiable topological loss for disentanglement learning. Our experiments have shown that the proposed TopDis loss improves disentanglement scores such as MIG, FactorVAE score, SAP score, and DCI disentanglement score with respect to state-of-the-art results while preserving the reconstruction quality. Our method works in an unsupervised manner, permitting us to apply it to problems without labeled factors of variation. The TopDis loss works even when factors of variation are correlated. Additionally, we show how to use the proposed topological loss to find disentangled directions in a trained GAN.

References in corpus (9)

• Semi-Supervised Learning with Deep Generative Models
• Deep Convolutional Inverse Graphics Network
• Learning Disentangled Representations with Semi-Supervised Deep Generative Models
• Disentangling Factors of Variation via Generative Entangling
• Regularizing Deep Multi-Task Networks using Orthogonal Gradients
• The Geometry of Deep Generative Image Models and its Applications
• A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation
• Disentanglement of Correlated Factors via Hausdorff Factorized Support
• Learning Topology-Preserving Data Representations