Does Momentum Change the Implicit Regularization on Separable Data?

Bohan Wang, Qi Meng, Huishuai Zhang +4

The momentum acceleration technique is widely adopted in many optimization algorithms. However, there is no theoretical answer on how the momentum affects the generalization perfor…