Parameter-free Clipped Gradient Descent Meets Polyak

Yuki Takezawa, Han Bao, Ryoma Sato +2

Gradient descent and its variants are de facto standard algorithms for training machine learning models. As gradient descent is sensitive to its hyperparameters, we need to tune th…