Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness

Abstract

The ability to learn disentangled representations that split underlying sources of variation in high dimensional, unstructured data is important for data efficient and robust use of neural networks. While various approaches aiming towards this goal have been proposed in recent times, a commonly accepted definition and validation procedure is missing. We provide a causal perspective on representation learning which covers disentanglement and domain shift robustness as special cases. Our causal framework allows us to introduce a new metric for the quantitative evaluation of deep latent variable models. We show how this metric can be estimated from labeled observational data and further provide an efficient estimation algorithm that scales linearly in the dataset size.

References in corpus (12)

• Auto-Encoding Variational Bayes
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Disentangling by Factorising
• Towards a Definition of Disentangled Representations
• Understanding disentangling in $\beta$-VAE
• Disentangling factors of variation in deep representations using adversarial training
• Invariant Models for Causal Transfer Learning
• Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
• Discovering Hidden Factors of Variation in Deep Networks
• Learning Deep Disentangled Embeddings with the F-Statistic Loss
• Disentangling Factors of Variation Using Few Labels
• Independently Controllable Features

Cited by in corpus (4)

• Causality for Machine Learning
• Causal Intervention for Weakly-Supervised Semantic Segmentation
• Disentangling User Interest and Conformity for Recommendation with Causal Embedding
• DEFT: Distilling Entangled Factors by Preventing Information Diffusion