Learning Independent Causal Mechanisms

Abstract

Statistical learning relies upon data sampled from a distribution, and we usually do not care what actually generated it in the first place. From the point of view of causal modeling, the structure of each distribution is induced by physical mechanisms that give rise to dependences between observables. Mechanisms, however, can be meaningful autonomous modules of generative models that make sense beyond a particular entailed data distribution, lending themselves to transfer between problems. We develop an algorithm to recover a set of independent (inverse) mechanisms from a set of transformed data points. The approach is unsupervised and based on a set of experts that compete for data generated by the mechanisms, driving specialization. We analyze the proposed method in a series of experiments on image data. Each expert learns to map a subset of the transformed data back to a reference distribution. The learned mechanisms generalize to novel domains. We discuss implications for transfer learning and links to recent trends in generative modeling.

ICML 2018

Cited by in corpus (49)

• Object-Centric Learning with Slot Attention
• Interventional Few-Shot Learning
• Causality for Machine Learning
• Deep Structural Causal Models for Tractable Counterfactual Inference
• Recurrent Independent Mechanisms
• Learning explanations that are hard to vary
• Causal Intervention for Weakly-Supervised Semantic Segmentation
• Counterfactuals uncover the modular structure of deep generative models
• Assaying Out-Of-Distribution Generalization in Transfer Learning
• Towards causal generative scene models via competition of experts
• Multi-Task Reinforcement Learning with Context-based Representations
• Learning Compositional Neural Programs with Recursive Tree Search and Planning
• Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progression
• Desiderata for Representation Learning: A Causal Perspective
• A Flexible Selection Scheme for Minimum-Effort Transfer Learning
• Semi-Supervised Learning, Causality and the Conditional Cluster Assumption
• Competitive Training of Mixtures of Independent Deep Generative Models
• Learning Group Structure and Disentangled Representations of Dynamical Environments
• Causal Influence Detection for Improving Efficiency in Reinforcement Learning
• Dynamic Inference with Neural Interpreters
• Modularity in Deep Learning: A Survey
• Size-Invariant Graph Representations for Graph Classification Extrapolations
• Modular Meta-Learning with Shrinkage
• Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning
• End-to-end optimized image compression with competition of prior distributions
• Visual Representation Learning Does Not Generalize Strongly Within the Same Domain
• Benchmarks, Algorithms, and Metrics for Hierarchical Disentanglement
• Shared Causal Paths underlying Alzheimer's dementia and Type 2 Diabetes
• Adaptive Skip Intervals: Temporal Abstraction for Recurrent Dynamical Models
• Latent Instrumental Variables as Priors in Causal Inference based on Independence of Cause and Mechanism
• Interventional Video Grounding with Dual Contrastive Learning
• Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective
• Transporting Causal Mechanisms for Unsupervised Domain Adaptation
• Independent mechanism analysis, a new concept?
• Neural Networks for Learning Counterfactual G-Invariances from Single Environments
• Beyond traditional assumptions in fair machine learning
• Fighting Copycat Agents in Behavioral Cloning from Observation Histories
• Deconfounded Score Method: Scoring DAGs with Dense Unobserved Confounding
• No Representation without Transformation
• Unsupervised Object Learning via Common Fate
• Towards Unbiased Visual Emotion Recognition via Causal Intervention
• Improving Weakly-supervised Object Localization via Causal Intervention
• Fast and Flexible Image Blind Denoising via Competition of Experts
• Adversarial defenses via a mixture of generators
• Efficiently Disentangle Causal Representations
• Learning Transferable Concepts in Deep Reinforcement Learning
• GalilAI: Out-of-Task Distribution Detection using Causal Active Experimentation for Safe Transfer RL
• Zero-shot generalization using cascaded system-representations
• Learning Spatial Relationships between Samples of Patent Image Shapes