Causality-aware counterfactual confounding adjustment for feature representations learned by deep models

Abstract

Causal modeling has been recognized as a potential solution to many challenging problems in machine learning (ML). Here, we describe how a recently proposed counterfactual approach developed to deconfound linear structural causal models can still be used to deconfound the feature representations learned by deep neural network (DNN) models. The key insight is that by training an accurate DNN using softmax activation at the classification layer, and then adopting the representation learned by the last layer prior to the output layer as our features, we have that, by construction, the learned features will fit well a (multi-class) logistic regression model, and will be linearly associated with the labels. As a consequence, deconfounding approaches based on simple linear models can be used to deconfound the feature representations learned by DNNs. We validate the proposed methodology using colored versions of the MNIST dataset. Our results illustrate how the approach can effectively combat confounding and improve model stability in the context of dataset shifts generated by selection biases.

References in corpus (12)

• SMOTE: Synthetic Minority Over-sampling Technique
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Invariant Risk Minimization
• Learning Representations for Counterfactual Inference
• Invariant Models for Causal Transfer Learning
• On Causal and Anticausal Learning
• Causality for Machine Learning
• Causally Regularized Learning with Agnostic Data Selection Bias
• Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions
• Learning Causal Structures Using Regression Invariance
• Causality-based tests to detect the influence of confounders on mobile health diagnostic applications: a comparison with restricted permutations
• Stable Prediction with Model Misspecification and Agnostic Distribution Shift

Cited by in corpus (3)

• Causality-aware counterfactual confounding adjustment as an alternative to linear residualization in anticausal prediction tasks based on linear learners
• Finding and Fixing Spurious Patterns with Explanations
• Stable predictions for health related anticausal prediction tasks affected by selection biases: the need to deconfound the test set features