On the Implicit Bias of Adam

Matias D. Cattaneo, Jason M. Klusowski, Boris Shigida

In previous literature, backward error analysis was used to find ordinary differential equations (ODEs) approximating the gradient descent trajectory. It was found that finite step…