Counterfactuals uncover the modular structure of deep generative models

Abstract

Deep generative models can emulate the perceptual properties of complex image datasets, providing a latent representation of the data. However, manipulating such representation to perform meaningful and controllable transformations in the data space remains challenging without some form of supervision. While previous work has focused on exploiting statistical independence to disentangle latent factors, we argue that such requirement is too restrictive and propose instead a non-statistical framework that relies on counterfactual manipulations to uncover a modular structure of the network composed of disentangled groups of internal variables. Experiments with a variety of generative models trained on complex image datasets show the obtained modules can be used to design targeted interventions. This opens the way to applications such as computationally efficient style transfer and the automated assessment of robustness to contextual changes in pattern recognition systems.

arXiv comment26 pages, 17 figures

References in corpus (14)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Neural Architecture Search with Reinforcement Learning
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Deep Convolutional Inverse Graphics Network
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Towards a Definition of Disentangled Representations
• Disentangling factors of variation in deep representations using adversarial training
• GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
• Learning Independent Causal Mechanisms
• Group invariance principles for causal generative models
• Examining CNN Representations with respect to Dataset Bias

Cited by in corpus (12)

• Causality for Machine Learning
• Causal Intervention for Weakly-Supervised Semantic Segmentation
• Causality Learning: A New Perspective for Interpretable Machine Learning
• Facial Action Unit Detection by Adaptively Constraining Self-Attention and Causally Deconfounding Sample
• Causal Interpretability for Machine Learning -- Problems, Methods and Evaluation
• Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests
• Interventions and Counterfactuals in Tractable Probabilistic Models: Limitations of Contemporary Transformations
• Bilateral Asymmetry Guided Counterfactual Generating Network for Mammogram Classification
• Interventional Video Grounding with Dual Contrastive Learning
• Difference-in-Differences: Bridging Normalization and Disentanglement in PG-GAN
• Improving Weakly-supervised Object Localization via Causal Intervention
• The Blessings of Unlabeled Background in Untrimmed Videos