Regularized Orthogonal Machine Learning for Nonlinear Semiparametric Models

Abstract

This paper proposes a Lasso-type estimator for a high-dimensional sparse parameter identified by a single index conditional moment restriction (CMR). In addition to this parameter, the moment function can also depend on a nuisance function, such as the propensity score or the conditional choice probability, which we estimate by modern machine learning tools. We first adjust the moment function so that the gradient of the future loss function is insensitive (formally, Neyman-orthogonal) with respect to the first-stage regularization bias, preserving the single index property. We then take the loss function to be an indefinite integral of the adjusted moment function with respect to the single index. The proposed Lasso estimator converges at the oracle rate, where the oracle knows the nuisance function and solves only the parametric problem. We demonstrate our method by estimating the short-term heterogeneous impact of Connecticut's Jobs First welfare reform experiment on women's welfare participation decision.

References in corpus (5)

• Semiparametric efficiency in GMM models with auxiliary data
• Adaptive Concentration of Regression Trees, with Application to Random Forests
• Discussion: The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
• Nonparametric estimation of causal heterogeneity under high-dimensional confounding
• High Dimensional M-Estimation with Missing Outcomes: A Semi-Parametric Framework

Cited by in corpus (1)

• Knowledge Distillation as Semiparametric Inference