For objective causal inference, design trumps analysis

Abstract

For obtaining causal inferences that are objective, and therefore have the best chance of revealing scientific truths, carefully designed and executed randomized experiments are generally considered to be the gold standard. Observational studies, in contrast, are generally fraught with problems that compromise any claim for objectivity of the resulting causal inferences. The thesis here is that observational studies have to be carefully designed to approximate randomized experiments, in particular, without examining any final outcome data. Often a candidate data set will have to be rejected as inadequate because of lack of data on key covariates, or because of lack of overlap in the distributions of key covariates between treatment and control groups, often revealed by careful propensity score analyses. Sometimes the template for the approximating randomized experiment will have to be altered, and the use of principal stratification can be helpful in doing this. These issues are discussed and illustrated using the framework of potential outcomes to define causal effects, which greatly clarifies critical issues.

Published in at http://dx.doi.org/10.1214/08-AOAS187 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (1)

• Estimating the Causal Effects of Marketing Interventions Using Propensity Score Methodology

Cited by in corpus (59)

• Balancing Covariates via Propensity Score Weighting
• Rerandomization to improve covariate balance in experiments
• Conceptualising Natural and Quasi Experiments in Public Health
• Minimal Dispersion Approximately Balancing Weights: Asymptotic Properties and Practical Considerations
• Causal inference in transportation safety studies: Comparison of potential outcomes and causal diagrams
• Causal Inference in Travel Demand Modeling (and the lack thereof)
• Study design in causal models
• FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal Inference
• Generalized Geographically Weighted Regression Model within a Modularized Bayesian Framework
• Covariate Balancing Propensity Score by Tailored Loss Functions
• To Adjust or Not to Adjust? Sensitivity Analysis of M-Bias and Butterfly-Bias
• Robust Estimation of Causal Effects via High-Dimensional Covariate Balancing Propensity Score
• Matching on Generalized Propensity Scores with Continuous Exposures
• Rebar: Reinforcing a Matching Estimator with Predictions from High-Dimensional Covariates
• Pairwise Sequential Randomization and Its Properties
• Labour by Design: Contributions of David Card, Joshua Angrist, and Guido Imbens
• Propensity Score Weighting for Causal Subgroup Analysis
• A Framework for Covariate Balance using Bregman Distances
• Principal stratification analysis using principal scores
• Varying impacts of letters of recommendation on college admissions: Approximate balancing weights for subgroup effects in observational studies
• Noise-Induced Randomization in Regression Discontinuity Designs
• Covariate adjustment in randomization-based causal inference for 2K factorial designs
• Uncertainty in the Design Stage of Two-Stage Bayesian Propensity Score Analysis
• Causal Relational Learning
• Causal Inference: A Missing Data Perspective
• Propensity Score Weighting for Covariate Adjustment in Randomized Clinical Trials
• A Bayesian Gaussian Process for Estimating a Causal Exposure Response Curve in Environmental Epidemiology
• From controlled to undisciplined data: estimating causal effects in the era of data science using a potential outcome framework
• A Generalizability Score for Aggregate Causal Effect
• Posterior Predictive Treatment Assignment for Estimating Causal Effects with Limited Overlap
• The Local Randomization Framework for Regression Discontinuity Designs: A Review and Some Extensions
• On finite-population Bayesian inferences for $2^K$ factorial designs with binary outcomes
• Estimating the average treatment effects of nutritional label use using subclassification with regression adjustment
• Interpretable Almost Matching Exactly for Causal Inference
• Probabilistic Analysis of Balancing Scores for Causal Inference
• A Bayesian Nonparametric Causal Model for Regression Discontinuity Designs
• Variable Selection in Causal Inference Using Penalization
• stratamatch: Prognostic ScoreStratification using a Pilot Design
• A Bayesian Nonparametric Hypothesis Testing Approach for Regression Discontinuity Designs
• Confounder selection via iterative graph expansion
• Sharpening randomization-based causal inference for $2^2$ factorial designs with binary outcomes
• Bias Variance Tradeoff in Analysis of Online Controlled Experiments
• Sensitivity analysis for bias due to a misclassfied confounding variable in marginal structural models
• Data-Driven Confounder Selection via Markov and Bayesian Networks
• On the probability of a causal inference is robust for internal validity
• Regularization and Hierarchical Prior Distributions for Adjustment with Health Care Claims Data: Rethinking Comorbidity Scores
• On randomization-based causal inference for matched-pair factorial designs
• The central role of Bayes theorem for joint estimation of causal effects and propensity scores
• Inference in Spatial Experiments with Interference using the SpatialEffect Package
• Treatment Effect Bias from Sample Snooping: Blinding Outcomes is Neither Necessary nor Sufficient
• Causal Deep Learning
• Re-Evaluating Strengthened-IV Designs: Asymptotic Efficiency, Bias Formula, and the Validity and Power of Sensitivity Analyses
• Leveraging Random Assignment to Impute Missing Covariates in Causal Studies
• Sharp bounds on the relative treatment effect for ordinal outcomes
• On Heavy-user Bias in A/B Testing
• Sources of Gain: Decomposing Performance in Conditional Average Dose Response Estimation
• Text as Causal Mediators: Research Design for Causal Estimates of Differential Treatment of Social Groups via Language Aspects
• Improved Neymanian analysis for $2^K$ factorial designs with binary outcomes
• Conceptualizing experimental controls using the potential outcomes framework