Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent

Avrajit Ghosh, He Lyu, Xitong Zhang +1

It is well known that the finite step-size ($h$) in Gradient Descent (GD) implicitly regularizes solutions to flatter minima. A natural question to ask is "Does the momentum parame…