On nearly assumption-free tests of nominal confidence interval coverage for causal parameters estimated by machine learning

Abstract

For many causal effect parameters of interest, doubly robust machine learning (DRML) estimators ${\hat{\psi }}_1$ are the state-of-the-art, incorporating the good prediction performance of machine learning; the decreased bias of doubly robust estimators; and the analytic tractability and bias reduction of sample splitting with cross fitting. Nonetheless, even in the absence of confounding by unmeasured factors, the nominal $(1-\alpha )$ Wald confidence interval ${\hat{\psi }}_1\pm z_{\alpha /2}\widehat{\mathsf{se}}[{\hat{\psi }}_1]$ may still undercover even in large samples, because the bias of ${\hat{\psi }}_1$ may be of the same or even larger order than its standard error of order $n^{-1/2}$. In this paper, we introduce essentially assumption-free tests that (i) can falsify the null hypothesis that the bias of ${\hat{\psi }}_1$ is of smaller order than its standard error, (ii) can provide an upper confidence bound on the true coverage of the Wald interval, and (iii) are valid under the null under no smoothness/sparsity assumptions on the nuisance parameters. The tests, which we refer to as \underline{A}ssumption \underline{F}ree \underline{E}mpirical \underline{C}overage \underline{T}ests (AFECTs), are based on a U-statistic that estimates part of the bias of ${\hat{\psi }}_1$.

arXiv commentSignificant updates from the previous version. In press in Statistical Science

References in corpus (17)

• Nonlinear shrinkage estimation of large-dimensional covariance matrices
• Exponential and moment inequalities for U-statistics
• Universal Inference
• Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality
• Efficient and Adaptive Linear Regression in Semi-Supervised Settings
• Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression
• Optimal Linear Shrinkage Estimator for Large Dimensional Precision Matrix
• Moment inequalities for U-statistics
• Approximation and Estimation for High-Dimensional Deep Learning Networks
• On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces
• How Well Generative Adversarial Networks Learn Distributions
• Estimation of the covariance structure of heavy-tailed distributions
• The Bayesian Analysis of Complex, High-Dimensional Models: Can It Be CODA?
• Selective machine learning of doubly robust functionals
• Adaptive Estimation of Nonparametric Functionals
• Technical Report: Higher Order Influence Functions and Minimax Estimation of Nonlinear Functionals
• Optimal estimation of variance in nonparametric regression with random design

Cited by in corpus (1)

• Floodgate: inference for model-free variable importance