Flatland: The Adventures of Gradient Descent with Large Step Sizes

Leonardo Galli, Curtis Fox, Wiebke Bartolomaeus +2

The training of neural networks often entails objective functions that are not globally $L$-smooth. For these functions, it is both theoretically and practically difficult to reply…