Lower Generalization Bounds for GD and SGD in Smooth Stochastic Convex Optimization

Peiyuan Zhang, Jiaye Teng, Jingzhao Zhang

This work studies the generalization error of gradient methods. More specifically, we focus on how training steps $T$ and step-size $\eta$ might affect generalization in smooth stocha…