A Primer on PAC-Bayesian Learning

Abstract

Generalised Bayesian learning algorithms are increasingly popular in machine learning, due to their PAC generalisation properties and flexibility. The present paper aims at providing a self-contained survey on the resulting PAC-Bayes framework and some of its main theoretical and algorithmic developments.

References in corpus (12)

• From $\epsilon$-entropy to KL-entropy: Analysis of minimum information complexity density estimation
• A Note on the PAC Bayesian Theorem
• Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory
• Fast learning rates in statistical inference through aggregation
• Fast Rates for General Unbounded Loss Functions: from ERM to Generalized Bayes
• Consistency of Variational Bayes Inference for Estimation and Model Selection in Mixtures
• A PAC-Bayesian Analysis of Randomized Learning with Application to Stochastic Gradient Descent
• Simpler PAC-Bayesian Bounds for Hostile Data
• A New PAC-Bayesian Perspective on Domain Adaptation
• A Bayesian Approach for Noisy Matrix Completion: Optimal Rate under General Sampling Distribution
• PAC-Bayesian bounds for the Gram matrix and least squares regression with a random design
• PAC-Bayesian High Dimensional Bipartite Ranking

Cited by in corpus (29)

• Bayesian Deep Learning and a Probabilistic Perspective of Generalization
• Generalized Variational Inference: Three arguments for deriving new Posteriors
• Generalization Bounds via Information Density and Conditional Information Density
• Tighter risk certificates for neural networks
• Subgroup Generalization and Fairness of Graph Neural Networks
• Gibbs posterior concentration rates under sub-exponential type losses
• Tighter expected generalization error bounds via Wasserstein distance
• On the role of data in PAC-Bayes bounds
• Generalization bounds for deep learning
• Convergence Rates of Variational Inference in Sparse Deep Learning
• PAC-Bayesian Bound for the Conditional Value at Risk
• A Generalization Bound for Online Variational Inference
• Latent Simplex Position Model: High Dimensional Multi-view Clustering with Uncertainty Quantification
• Generalized Posteriors in Approximate Bayesian Computation
• Stopping Criterion for Active Learning Based on Error Stability
• Still no free lunches: the price to pay for tighter PAC-Bayes bounds
• PAC-Bayesian Contrastive Unsupervised Representation Learning
• Novel Change of Measure Inequalities with Applications to PAC-Bayesian Bounds and Monte Carlo Estimation
• Adaptive Task Sampling for Meta-Learning
• Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks
• Robust Bayesian Inference for Discrete Outcomes with the Total Variation Distance
• Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Nets
• Risk-Monotonicity in Statistical Learning
• Graph Signal Processing over a Probability Space of Shift Operators
• Statistical Inference for Bayesian Risk Minimization via Exponentially Tilted Empirical Likelihood
• Practical calibration of the temperature parameter in Gibbs posteriors
• Unifying Variational Inference and PAC-Bayes for Supervised Learning that Scales
• Multi-task Federated Edge Learning (MtFEEL) in Wireless Networks
• Signal processing with a distribution of graph operators