A Note on the PAC Bayesian Theorem

Abstract

We prove general exponential moment inequalities for averages of [0,1]-valued iid random variables and use them to tighten the PAC Bayesian Theorem. The logarithmic dependence on the sample count in the enumerator of the PAC Bayesian bound is halved.

9 pages

Cited by in corpus (42)

• Fairness in Machine Learning
• Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithm
• Empirical Risk Minimization under Fairness Constraints
• User-friendly introduction to PAC-Bayes bounds
• A Primer on PAC-Bayesian Learning
• Tighter risk certificates for neural networks
• A New PAC-Bayesian Perspective on Domain Adaptation
• PAC-Bayes under potentially heavy tails
• PAC-Bayes and Domain Adaptation
• PAC-Bayes with Backprop
• Information Complexity and Generalization Bounds
• PAC-Bayesian Theory Meets Bayesian Inference
• PAC-Bayesian Bound for the Conditional Value at Risk
• Generalization bounds for deep learning
• Generalization Guarantees for Imitation Learning
• Data-dependent PAC-Bayes priors via differential privacy
• Latent Simplex Position Model: High Dimensional Multi-view Clustering with Uncertainty Quantification
• PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classes
• PAC-Bayesian Theorems for Domain Adaptation with Specialization to Linear Classifiers
• PAC-Bayes Analysis Beyond the Usual Bounds
• Efron-Stein PAC-Bayesian Inequalities
• Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks
• Upper and Lower Bounds on the Performance of Kernel PCA
• PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparison
• Learning Partially Known Stochastic Dynamics with Empirical PAC Bayes
• Probably Approximately Correct Vision-Based Planning using Motion Primitives
• Bregman Divergence Bounds and Universality Properties of the Logarithmic Loss
• Wide stochastic networks: Gaussian limit and PAC-Bayesian training
• Learning via Wasserstein-Based High Probability Generalisation Bounds
• How Tight Can PAC-Bayes be in the Small Data Regime?
• Pair-Wise Cluster Analysis
• Chebyshev-Cantelli PAC-Bayes-Bennett Inequality for the Weighted Majority Vote
• Generalisation under gradient descent via deterministic PAC-Bayes
• Split-kl and PAC-Bayes-split-kl Inequalities for Ternary Random Variables
• Unifying Variational Inference and PAC-Bayes for Supervised Learning that Scales
• PAC-Bayes-Chernoff bounds for unbounded losses
• Comparing Comparators in Generalization Bounds
• Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stability
• On the Generalization of the C-Bound to Structured Output Ensemble Methods
• A note on regularised NTK dynamics with an application to PAC-Bayesian training
• Multi-class Probabilistic Bounds for Self-learning
• A combinatorial conjecture from PAC-Bayesian machine learning