Latent Causal Invariant Model

Abstract

Current supervised learning can learn spurious correlation during the data-fitting process, imposing issues regarding interpretability, out-of-distribution (OOD) generalization, and robustness. To avoid spurious correlation, we propose a Latent Causal Invariance Model (LaCIM) which pursues causal prediction. Specifically, we introduce latent variables that are separated into (a) output-causative factors and (b) others that are spuriously correlated to the output via confounders, to model the underlying causal factors. We further assume the generating mechanisms from latent space to observed data to be causally invariant. We give the identifiable claim of such invariance, particularly the disentanglement of output-causative factors from others, as a theoretical guarantee for precise inference and avoiding spurious correlation. We propose a Variational-Bayesian-based method for estimation and to optimize over the latent space for prediction. The utility of our approach is verified by improved interpretability, prediction power on various OOD scenarios (including healthcare) and robustness on security.

References in corpus (24)

• Auto-Encoding Variational Bayes
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Understanding deep learning requires rethinking generalization
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• Generalizing Across Domains via Cross-Gradient Training
• Invariant Risk Minimization
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
• Improving Deep Learning using Generic Data Augmentation
• On Learning Invariant Representation for Domain Adaptation
• Invariant Models for Causal Transfer Learning
• Causality for Machine Learning
• The Consciousness Prior
• DIVA: Domain Invariant Variational Autoencoders
• Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions
• PatchShuffle Regularization
• Support and Invertibility in Domain-Invariant Representations
• Unshuffling Data for Improved Generalization
• Few-shot Domain Adaptation by Causal Mechanism Transfer
• Selecting Data Augmentation for Simulating Interventions
• I-SPEC: An End-to-End Framework for Learning Transportable, Shift-Stable Models
• Leveraging directed causal discovery to detect latent common causes
• Robust Learning via Cause-Effect Models

Cited by in corpus (4)

• Nonlinear Invariant Risk Minimization: A Causal Approach
• Causal Hidden Markov Model for Time Series Disease Forecasting
• Context-LGM: Leveraging Object-Context Relation for Context-Aware Object Recognition
• $\beta$-Intact-VAE: Identifying and Estimating Causal Effects under Limited Overlap