Counterfactual Cross-Validation: Stable Model Selection Procedure for Causal Inference Models
arXiv:1909.05299
Abstract
We study the model selection problem in conditional average treatment effect (CATE) prediction. Unlike previous works on this topic, we focus on preserving the rank order of the performance of candidate CATE predictors to enable accurate and stable model selection. To this end, we analyze the model performance ranking problem and formulate guidelines to obtain a better evaluation metric. We then propose a novel metric that can identify the ranking of the performance of CATE predictors with high confidence. Empirical evaluations demonstrate that our metric outperforms existing metrics in both model selection and hyperparameter tuning tasks.
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Domain Adaptation: Learning Bounds and Algorithms
- Fast Computation of Wasserstein Barycenters
- Doubly Robust Policy Evaluation and Learning
- Causal Effect Inference with Deep Latent-Variable Models
- Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networks
- Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects
- A comparison of methods for model selection when estimating individual treatment effects
- A Survey on Causal Inference