Causally Regularized Learning with Agnostic Data Selection Bias

Abstract

Most of previous machine learning algorithms are proposed based on the i.i.d. hypothesis. However, this ideal assumption is often violated in real applications, where selection bias may arise between training and testing process. Moreover, in many scenarios, the testing data is not even available during the training process, which makes the traditional methods like transfer learning infeasible due to their need on prior of test distribution. Therefore, how to address the agnostic selection bias for robust model learning is of paramount importance for both academic research and real applications. In this paper, under the assumption that causal relationships among variables are robust across domains, we incorporate causal technique into predictive modeling and propose a novel Causally Regularized Logistic Regression (CRLR) algorithm by jointly optimize global confounder balancing and weighted logistic regression. Global confounder balancing helps to identify causal features, whose causal effect on outcome are stable across domains, then performing logistic regression on those causal features constructs a robust predictive model against the agnostic bias. To validate the effectiveness of our CRLR algorithm, we conduct comprehensive experiments on both synthetic and real world datasets. Experimental results clearly demonstrate that our CRLR algorithm outperforms the state-of-the-art methods, and the interpretability of our method can be fully depicted by the feature visualization.

Oral paper of 2018 ACM Multimedia Conference (MM'18)

References in corpus (6)

• Power-law distributions in empirical data
• Matching Methods for Causal Inference: A Review and a Look Forward
• Learning Transferable Features with Deep Adaptation Networks
• Unsupervised Domain Adaptation by Backpropagation
• Domain Generalization via Invariant Feature Representation
• Deep Transfer Learning with Joint Adaptation Networks

Cited by in corpus (15)

• Towards Out-Of-Distribution Generalization: A Survey
• Learning Causal Semantic Representation for Out-of-Distribution Prediction
• When is invariance useful in an Out-of-Distribution Generalization problem ?
• Learning causal representations for robust domain adaptation
• Towards Non-I.I.D. Image Classification: A Dataset and Baselines
• Reformulating CTR Prediction: Learning Invariant Feature Interactions for Recommendation
• COCO-CN for Cross-Lingual Image Tagging, Captioning and Retrieval
• Generalizing Graph Neural Networks on Out-Of-Distribution Graphs
• Causality-aware counterfactual confounding adjustment for feature representations learned by deep models
• Stable Prediction with Model Misspecification and Agnostic Distribution Shift
• Analysis and Applications of Class-wise Robustness in Adversarial Training
• Decorrelated Clustering with Data Selection Bias
• Contrastive ACE: Domain Generalization Through Alignment of Causal Mechanisms
• A Generalization Theory based on Independent and Task-Identically Distributed Assumption
• Distributionally Robust Learning with Stable Adversarial Training