Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity

Abstract

We study the problem of aggregation under the squared loss in the model of regression with deterministic design. We obtain sharp PAC-Bayesian risk bounds for aggregates defined via exponential weights, under general assumptions on the distribution of errors and on the functions to aggregate. We then apply these results to derive sparsity oracle inequalities.

References in corpus (5)

• Simultaneous analysis of Lasso and Dantzig selector
• The Skorokhod embedding problem and its offspring
• Sparsity oracle inequalities for the Lasso
• Aggregation for Gaussian regression
• From $\epsilon$-entropy to KL-entropy: Analysis of minimum information complexity density estimation

Cited by in corpus (29)

• Sparse recovery under matrix uncertainty
• Sparse Regression Learning by Aggregation and Langevin Monte-Carlo
• Empirical Bayes posterior concentration in sparse high-dimensional linear models
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Sparse Estimation by Exponential Weighting
• Deviation optimal learning using greedy Q-aggregation
• Sharp Oracle Inequalities for Aggregation of Affine Estimators
• 1-bit Matrix Completion: PAC-Bayesian Analysis of a Variational Approximation
• Mirror averaging with sparsity priors
• Bayesian methods for low-rank matrix estimation: short survey and theoretical study
• Pac-bayesian bounds for sparse regression estimation with exponential weights
• A Bayesian Approach for Noisy Matrix Completion: Optimal Rate under General Sampling Distribution
• Pseudo-Bayesian Quantum Tomography with Rank-adaptation
• PAC-Bayesian Estimation and Prediction in Sparse Additive Models
• Optimal learning with $Q$-aggregation
• Performance of empirical risk minimization in linear aggregation
• Generalized Geographically Weighted Regression Model within a Modularized Bayesian Framework
• Aggregation of predictors for nonstationary sub-linear processes and online adaptive forecasting of time varying autoregressive processes
• An Oracle Inequality for Quasi-Bayesian Non-Negative Matrix Factorization
• Empirical risk minimization is optimal for the convex aggregation problem
• Exponential weights in multivariate regression and a low-rankness favoring prior
• PAC-Bayesian High Dimensional Bipartite Ranking
• From bilinear regression to inductive matrix completion: a quasi-Bayesian analysis
• A reduced-rank approach to predicting multiple binary responses through machine learning
• High-dimensional sparse classification using exponential weighting with empirical hinge loss
• A Quasi-Bayesian Perspective to Online Clustering
• Bernstein - von Mises theorem and misspecified models: a review
• Misclassification bounds for PAC-Bayesian sparse deep learning
• An efficient adaptive MCMC algorithm for Pseudo-Bayesian quantum tomography