Generalization Bounds of Stochastic Gradient Descent in Homogeneous Neural Networks

Wenquan Ma, Yang Sui, Jiaye Teng +3

Algorithmic stability is among the most potent techniques in generalization analysis. However, its derivation usually requires a stepsize ${\eta }_t=\mathcal{O}(1/t)$ under non-convex…