High-dimensional sparse classification using exponential weighting with empirical hinge loss

Abstract

In this study, we address the problem of high-dimensional binary classification. Our proposed solution involves employing an aggregation technique founded on exponential weights and empirical hinge loss. Through the employment of a suitable sparsity-inducing prior distribution, we demonstrate that our method yields favorable theoretical results on prediction error. The efficiency of our procedure is achieved through the utilization of Langevin Monte Carlo, a gradient-based sampling approach. To illustrate the effectiveness of our approach, we conduct comparisons with the logistic Lasso on simulated data and a real dataset. Our method frequently demonstrates superior performance compared to the logistic Lasso.

References in corpus (16)

• High-dimensional classification using features annealed independence rules
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• A General Framework for Updating Belief Distributions
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity
• Sparse Regression Learning by Aggregation and Langevin Monte-Carlo
• Calibrating general posterior credible regions
• Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithm
• On the properties of variational approximations of Gibbs posteriors
• User-friendly introduction to PAC-Bayes bounds
• On optimality of Bayesian testimation in the normal means problem
• 1-bit Matrix Completion: PAC-Bayesian Analysis of a Variational Approximation
• Mirror averaging with sparsity priors
• A Bayesian Approach for Noisy Matrix Completion: Optimal Rate under General Sampling Distribution
• Pseudo-Bayesian Quantum Tomography with Rank-adaptation
• Exponential weights in multivariate regression and a low-rankness favoring prior