Maximum Regularized Likelihood Estimators: A General Prediction Theory and Applications

Abstract

Maximum regularized likelihood estimators (MRLEs) are arguably the most established class of estimators in high-dimensional statistics. In this paper, we derive guarantees for MRLEs in Kullback-Leibler divergence, a general measure of prediction accuracy. We assume only that the densities have a convex parametrization and that the regularization is definite and positive homogenous. The results thus apply to a very large variety of models and estimators, such as tensor regression and graphical models with convex and non-convex regularized methods. A main conclusion is that MRLEs are broadly consistent in prediction - regardless of whether restricted eigenvalues or similar conditions hold.

References in corpus (13)

• Tensor Regression with Applications in Neuroimaging Data Analysis
• On the Prediction Performance of the Lasso
• A new perspective on least squares under convex constraint
• Exponential Screening and optimal rates of sparse estimation
• STORE: Sparse Tensor Response Regression and Neuroimaging Analysis
• Assumptionless consistency of the Lasso
• Low-rank spectral optimization via gauge duality
• Local and Global Inference for High Dimensional Nonparanormal Graphical Models
• Tucker Tensor Regression and Neuroimaging Analysis
• Integrating Additional Knowledge Into Estimation of Graphical Models
• Graphical Models for Discrete and Continuous Data
• Fast-rate and optimistic-rate error bounds for L1-regularized regression
• Identifying Graphical Models