High Dimensional M-Estimation with Missing Outcomes: A Semi-Parametric Framework

Abstract

We consider high dimensional $M$-estimation in settings where the response $Y$ is possibly missing at random and the covariates $\mathbf{X}\in {\mathbb{R}}^p$ can be high dimensional compared to the sample size $n$. The parameter of interest ${\mathbf{\theta }}_0\in {\mathbb{R}}^d$ is defined as the minimizer of the risk of a convex loss, under a fully non-parametric model, and ${\mathbf{\theta }}_0$ itself is high dimensional which is a key distinction from existing works. Standard high dimensional regression and series estimation with possibly misspecified models and missing $Y$ are included as special cases, as well as their counterparts in causal inference using 'potential outcomes'. Assuming ${\mathbf{\theta }}_0$ is $s$-sparse ($s\ll n$), we propose an $L_1$-regularized debiased and doubly robust (DDR) estimator of ${\mathbf{\theta }}_0$ based on a high dimensional adaptation of the traditional double robust (DR) estimator's construction. Under mild tail assumptions and arbitrarily chosen (working) models for the propensity score (PS) and the outcome regression (OR) estimators, satisfying only some high-level conditions, we establish finite sample performance bounds for the DDR estimator showing its (optimal) $L_2$ error rate to be $\sqrt{s(\log d)/n}$ when both models are correct, and its consistency and DR properties when only one of them is correct. Further, when both the models are correct, we propose a desparsified version of our DDR estimator that satisfies an asymptotic linear expansion and facilitates inference on low dimensional components of ${\mathbf{\theta }}_0$. Finally, we discuss various of choices of high dimensional parametric/semi-parametric working models for the PS and OR estimators. All results are validated via detailed simulations.

34 pages, 4 tables; (Supplement: 58 pages, 10 tables);

References in corpus (11)

• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity
• Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations
• Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression
• High-dimensional inference in misspecified linear models
• Sparse single-index model
• A unifying approach for doubly-robust ${\ell }_1$ regularized estimation of causal contrasts
• De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers
• Honest data-adaptive inference for the average treatment effect under model misspecification using penalised bias-reduced double-robust estimation
• Structured signal recovery from non-linear and heavy-tailed measurements
• Structured Recovery with Heavy-tailed Measurements: A Thresholding Procedure and Optimal Rates

Cited by in corpus (5)

• Double Robust Semi-Supervised Inference for the Mean: Selection Bias under MAR Labeling with Decaying Overlap
• Surrogate Assisted Semi-supervised Inference for High Dimensional Risk Prediction
• Regularized Orthogonal Machine Learning for Nonlinear Semiparametric Models
• High-dimensional Inference for Dynamic Treatment Effects
• Dynamic treatment effects: high-dimensional inference under model misspecification