PAC-Bayes Un-Expected Bernstein Inequality

Zakaria Mhammedi, Peter D. Grunwald, Benjamin Guedj

We present a new PAC-Bayesian generalization bound. Standard bounds contain a $\sqrt{L_n \cdot \KL/n}$ complexity term which dominates unless $L_n$, the empirical error of the lear…