CASTLE: Regularization via Auxiliary Causal Graph Discovery

Abstract

Regularization improves generalization of supervised models to out-of-sample data. Prior works have shown that prediction in the causal direction (effect from cause) results in lower testing error than the anti-causal direction. However, existing regularization methods are agnostic of causality. We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables. CASTLE learns the causal directed acyclical graph (DAG) as an adjacency matrix embedded in the neural network's input layers, thereby facilitating the discovery of optimal predictors. Furthermore, CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features. We provide a theoretical generalization bound for our approach and conduct experiments on a plethora of synthetic and real publicly available datasets demonstrating that CASTLE consistently leads to better out-of-sample predictions as compared to other popular benchmark regularizers.

References in corpus (18)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Improving neural networks by preventing co-adaptation of feature detectors
• On the difficulty of training Recurrent Neural Networks
• On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
• Deep Label Distribution Learning with Label Ambiguity
• Dropout Training as Adaptive Regularization
• DAGs with NO TEARS: Continuous Optimization for Structure Learning
• Invariant Models for Causal Transfer Learning
• ${\ell }_0$-penalized maximum likelihood for sparse directed acyclic graphs
• The Benefit of Multitask Representation Learning
• High-dimensional learning of linear causal networks via inverse covariance estimation
• Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions
• CXPlain: Causal Explanations for Model Interpretation under Uncertainty
• Anchor regression: heterogeneous data meets causality
• Learning Directed Acyclic Graphs with Penalized Neighbourhood Regression
• Learning Sparse Nonparametric DAGs
• Estimating Causal Direction and Confounding of Two Discrete Variables
• Causal Regularization

Cited by in corpus (6)

• CausPref: Causal Preference Learning for Out-of-Distribution Recommendation
• Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph
• Discovering Dynamic Causal Space for DAG Structure Learning
• Divide and Rule: Recurrent Partitioned Network for Dynamic Processes
• Bounding Random Test Set Size with Computational Learning Theory
• Inference of Causal Effects when Control Variables are Unknown