activity
20232026
most citedFaster Convergence for Transformer Fine-tuning with Line Search Methods

2 citations · 3 across the 3 of their papers we have counts for

collaborators
Showing math.OCShow all

1 paper · 1 filter

math.OC2023★ 1 cited

Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models

Leonardo Galli, Holger Rauhut, Mark Schmidt

Recent works have shown that line search methods can speed up Stochastic Gradient Descent (SGD) and Adam in modern over-parameterized settings. However, existing line searches may…