Independent mechanism analysis, a new concept?
arXiv:2106.05200
Abstract
Independent component analysis provides a principled framework for unsupervised representation learning, with solid theory on the identifiability of the latent code that generated the data, given only observations of mixtures thereof. Unfortunately, when the mixing is nonlinear, the model is provably nonidentifiable, since statistical independence alone does not sufficiently constrain the problem. Identifiability can be recovered in settings where additional, typically observed variables are included in the generative process. We investigate an alternative path and consider instead including assumptions reflecting the principle of independent causal mechanisms exploited in the field of causality. Specifically, our approach is motivated by thinking of each source as independently influencing the mixing process. This gives rise to a framework which we term independent mechanism analysis. We provide theoretical and empirical evidence that our approach circumvents a number of nonidentifiability issues arising in nonlinear blind source separation.
arXiv commentNeurIPS 2021 final camera-ready version
References in corpus (23)
- Scikit-learn: Machine Learning in Python
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Distinguishing cause from effect using observational data: methods and benchmarks
- Invariant Risk Minimization
- Faster independent component analysis by preconditioning with Hessian approximations
- On the Identifiability of the Post-Nonlinear Causal Model
- Identifiability of Gaussian structural equation models with equal error variances
- Residual Flows for Invertible Generative Modeling
- Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
- Invariant Models for Causal Transfer Learning
- Inferring deterministic causal relations
- Learning Independent Causal Mechanisms
- Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions
- Anchor regression: heterogeneous data meets causality
- Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search
- Deconfounding Reinforcement Learning in Observational Settings
- Testing whether linear equations are causal: A free probability theory approach
- Telling cause from effect in deterministic linear dynamical systems
- Towards causal generative scene models via competition of experts
- Group invariance principles for causal generative models
- Causal Markov condition for submodular information measures
- A Spectral Regularizer for Unsupervised Disentanglement
- Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation