PAC-Bayes bounds for stable algorithms with instance-dependent priors
arXiv:1806.06827
Abstract
PAC-Bayes bounds have been proposed to get risk estimates based on a training sample. In this paper the PAC-Bayes approach is combined with stability of the hypothesis learned by a Hilbert space valued algorithm. The PAC-Bayes setting is used with a Gaussian prior centered at the expected output. Thus a novelty of our paper is using priors defined in terms of the data-generating distribution. Our main result estimates the risk of the randomized algorithm in terms of the hypothesis stability coefficients. We also provide a new bound for the SVM classifier, which is compared to other known bounds experimentally. Ours appears to be the first stability-based bound that evaluates to non-trivial values.
arXiv comment16 pages, discussion of theory and experiments in the main body, detailed proofs and experimental details in the appendices
Cited by in corpus (7)
- Meta-Learning PAC-Bayes Priors in Model Averaging
- PAC-Bayes Analysis Beyond the Usual Bounds
- Efron-Stein PAC-Bayesian Inequalities
- An Optimal Transport View on Generalization
- PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparison
- Understanding Deep Architectures with Reasoning Layer
- Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stability