2 citations · 3 across the 3 of their papers we have counts for
3 papers
Flatland: The Adventures of Gradient Descent with Large Step Sizes
Leonardo Galli, Curtis Fox, Wiebke Bartolomaeus +2
The training of neural networks often entails objective functions that are not globally -smooth. For these functions, it is both theoretically and practically difficult to reply…
Faster Convergence for Transformer Fine-tuning with Line Search Methods
Philip Kenneweg, Leonardo Galli, Tristan Kenneweg +1
Recent works have shown that line search methods greatly increase performance of traditional stochastic gradient descent methods on a variety of datasets and architectures [1], [2]…
Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models
Leonardo Galli, Holger Rauhut, Mark Schmidt
Recent works have shown that line search methods can speed up Stochastic Gradient Descent (SGD) and Adam in modern over-parameterized settings. However, existing line searches may…