How Many Iterations are Sufficient for Semiparametric Estimation?

Abstract

A common practice in obtaining a semiparametric efficient estimate is through iteratively maximizing the (penalized) log-likelihood w.r.t. its Euclidean parameter and functional nuisance parameter via Newton-Raphson algorithm. The purpose of this paper is to provide a formula in calculating the minimal number of iterations $k^{\ast }$ needed to produce an efficient estimate ${\hat{\theta }}_n^{(k^{\ast })}$ from a theoretical point of view. We discover that (a) $k^{\ast }$ depends on the convergence rates of the initial estimate and nuisance estimate; (b) more than $k^{\ast }$ iterations, i.e., $k$, will only improve the higher order asymptotic efficiency of ${\hat{\theta }}_n^{(k)}$; (c) $k^{\ast }$ iterations are also sufficient for recovering the estimation sparsity in high dimensional data. These general conclusions hold, in particular, when the nuisance parameter is not estimable at root-n rate, and apply to semiparametric models estimated under various regularizations, e.g., kernel or penalized estimation. This paper provides a first general theoretical justification for the "one-/two-step iteration" phenomena observed in the literature, and may be useful in reducing the bootstrap computational cost for the semiparametric models.

42 pages, submitted to the Annals of Statistics

References in corpus (12)

• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al
• Two likelihood-based semiparametric estimation methods for panel count data with covariates
• Estimation of a semiparametric transformation model