PAC-Bayesian Bounds for Randomized Empirical Risk Minimizers

Abstract

The aim of this paper is to generalize the PAC-Bayesian theorems proved by Catoni in the classification setting to more general problems of statistical inference. We show how to control the deviations of the risk of randomized estimators. A particular attention is paid to randomized estimators drawn in a small neighborhood of classical estimators, whose study leads to control the risk of the latter. These results allow to bound the risk of very general estimation procedures, as well as to perform model selection.

Cited by in corpus (22)

• A General Framework for Updating Belief Distributions
• Sparse Regression Learning by Aggregation and Langevin Monte-Carlo
• User-friendly introduction to PAC-Bayes bounds
• Robust empirical mean Estimators
• Sparse single-index model
• Mirror averaging with sparsity priors
• Pac-bayesian bounds for sparse regression estimation with exponential weights
• Gibbs posterior concentration rates under sub-exponential type losses
• PAC-Bayesian bounds for the Gram matrix and least squares regression with a random design
• PAC-Bayesian Estimation and Prediction in Sparse Additive Models
• Gibbs posterior inference on multivariate quantiles
• PAC-Bayesian aggregation and multi-armed bandits
• PAC-Bayesian AUC classification and scoring
• Bayesian matrix completion: prior specification
• Linear regression through PAC-Bayesian truncation
• PAC-Bayesian High Dimensional Bipartite Ranking
• Risk bounds in linear regression through PAC-Bayesian truncation
• Fast rates in learning with dependent observations
• Non-exponentially weighted aggregation: regret bounds for unbounded loss functions
• The exponentially weighted average forecaster in geodesic spaces of non-positive curvature
• Practical calibration of the temperature parameter in Gibbs posteriors
• Statistical Inference for Bayesian Risk Minimization via Exponentially Tilted Empirical Likelihood