Kullback-Leibler aggregation and misspecified generalized linear models

Abstract

In a regression setup with deterministic design, we study the pure aggregation problem and introduce a natural extension from the Gaussian distribution to distributions in the exponential family. While this extension bears strong connections with generalized linear models, it does not require identifiability of the parameter or even that the model on the systematic component is true. It is shown that this problem can be solved by constrained and/or penalized likelihood maximization and we derive sharp oracle inequalities that hold both in expectation and with high probability. Finally all the bounds are proved to be optimal in a minimax sense.

Published in at http://dx.doi.org/10.1214/11-AOS961 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (5)

• Aggregation for Gaussian regression
• Best subset selection, persistence in high-dimensional statistical learning and optimization under $l_1$ constraint
• Sparse Estimation by Exponential Weighting
• Spatial aggregation of local likelihood estimates with applications to classification
• Pac-bayesian bounds for sparse regression estimation with exponential weights

Cited by in corpus (11)

• Sparse Estimation by Exponential Weighting
• Deviation optimal learning using greedy Q-aggregation
• Sharp Oracle Inequalities for Aggregation of Affine Estimators
• Optimal learning with $Q$-aggregation
• Optimal bounds for aggregation of affine estimators
• Empirical risk minimization is optimal for the convex aggregation problem
• Structured Estimation in Nonparameteric Cox Model
• Optimal exponential bounds for aggregation of density estimators
• Empirical Bayes inference in sparse high-dimensional generalized linear models
• Minimax Rates for Conditional Density Estimation via Empirical Entropy
• An adaptive multiclass nearest neighbor classifier