Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects

Abstract

Practitioners in diverse fields such as healthcare, economics and education are eager to apply machine learning to improve decision making. The cost and impracticality of performing experiments and a recent monumental increase in electronic record keeping has brought attention to the problem of evaluating decisions based on non-experimental observational data. This is the setting of this work. In particular, we study estimation of individual-level causal effects, such as a single patient's response to alternative medication, from recorded contexts, decisions and outcomes. We give generalization bounds on the error in estimated effects based on distance measures between groups receiving different treatments, allowing for sample re-weighting. We provide conditions under which our bound is tight and show how it relates to results for unsupervised domain adaptation. Led by our theoretical results, we devise representation learning algorithms that minimize our bound, by regularizing the representation's induced treatment group distance, and encourage sharing of information between treatment groups. We extend these algorithms to simultaneously learn a weighted representation to further reduce treatment group distances. Finally, an experimental evaluation on real and synthetic data shows the value of our proposed representation architecture and regularization scheme.

References in corpus (18)

• Learning Transferable Features with Deep Adaptation Networks
• Recursive Partitioning for Heterogeneous Causal Effects
• Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning
• Domain Adaptation: Learning Bounds and Algorithms
• Fast Computation of Wasserstein Barycenters
• Causal Effect Inference with Deep Latent-Variable Models
• Stochastic Optimization for Large-scale Optimal Transport
• Counterfactual Risk Minimization: Learning from Logged Bandit Feedback
• On integral probability metrics, ϕ-divergences and binary classification
• Causal Discovery from Changes
• Adapting Neural Networks for the Estimation of Treatment Effects
• Causal Inference by Identification of Vector Autoregressive Processes with Hidden Components
• Learning Weighted Representations for Generalization Across Designs
• Generalized Optimal Matching Methods for Causal Inference
• Overlap in Observational Studies with High-Dimensional Covariates
• Large-scale Validation of Counterfactual Learning Methods: A Test-Bed
• Benchmarking Framework for Performance-Evaluation of Causal Inference Analysis
• A Framework for Optimal Matching for Causal Inference

Cited by in corpus (10)

• Desiderata for Representation Learning: A Causal Perspective
• Fundamental Limits and Tradeoffs in Invariant Representation Learning
• SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data
• Counterfactual Representation Learning with Balancing Weights
• Counterfactual Cross-Validation: Stable Model Selection Procedure for Causal Inference Models
• GraphITE: Estimating Individual Effects of Graph-structured Treatments
• A pragmatic approach to estimating average treatment effects from EHR data: the effect of prone positioning on mechanically ventilated COVID-19 patients
• $\beta$-Intact-VAE: Identifying and Estimating Causal Effects under Limited Overlap
• Estimating Potential Outcome Distributions with Collaborating Causal Networks
• Causal Forecasting:Generalization Bounds for Autoregressive Models