Optimizing Information-theoretical Generalization Bounds via Anisotropic Noise in SGLD

Bohan Wang, Huishuai Zhang, Jieyu Zhang +3

Recently, the information-theoretical framework has been proven to be able to obtain non-vacuous generalization bounds for large models trained by Stochastic Gradient Langevin Dyna…