A new scope of penalized empirical likelihood with high-dimensional estimating equations

Abstract

Statistical methods with empirical likelihood (EL) are appealing and effective especially in conjunction with estimating equations through which useful data information can be adaptively and flexibly incorporated. It is also known in the literature that EL approaches encounter difficulties when dealing with problems having high-dimensional model parameters and estimating equations. To overcome the challenges, we begin our study with a careful investigation on high-dimensional EL from a new scope targeting at estimating a high-dimensional sparse model parameters. We show that the new scope provides an opportunity for relaxing the stringent requirement on the dimensionality of the model parameter. Motivated by the new scope, we then propose a new penalized EL by applying two penalty functions respectively regularizing the model parameters and the associated Lagrange multipliers in the optimizations of EL. By penalizing the Lagrange multiplier to encourage its sparsity, we show that drastic dimension reduction in the number of estimating equations can be effectively achieved without compromising the validity and consistency of the resulting estimators. Most attractively, such a reduction in dimensionality of estimating equations is actually equivalent to a selection among those high-dimensional estimating equations, resulting in a highly parsimonious and effective device for high-dimensional sparse model parameters. Allowing both the dimensionalities of model parameters and estimating equations growing exponentially with the sample size, our theory demonstrates that the estimator from our new penalized EL is sparse and consistent with asymptotically normally distributed nonzero components. Numerical simulations and a real data analysis show that the proposed penalized EL works promisingly.

References in corpus (7)

• Nearly unbiased variable selection under minimax concave penalty
• Pathwise coordinate optimization
• Coordinate descent algorithms for lasso penalized regression
• A unified approach to model selection and sparse recovery using regularized least squares
• Extending the scope of empirical likelihood
• A penalized empirical likelihood method in high dimensions
• Local independence feature screening for nonparametric and semiparametric models by marginal empirical likelihood

Cited by in corpus (7)

• High-dimensional empirical likelihood inference
• Testing the martingale difference hypothesis in high dimension
• Elastic Integrative Analysis of Randomized Trial and Real-World Data for Treatment Heterogeneity Estimation
• Culling the herd of moments with penalized empirical likelihood
• Empirical Likelihood Inference for Area under the ROC Curve using Ranked Set Samples
• Maximum Likelihood under constraints: Degeneracies and Random Critical Points
• Semiparametric efficient estimation of structural nested mean models with irregularly spaced observations