32 citations · 95 across the 15 of their papers we have counts for
Showing 2020Show all
2 papers · 1 filter
cs.LG2020
Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian
Jack Parker-Holder, Luke Metz, Cinjon Resnick +6
Over the last decade, a single algorithm has changed many facets of our lives - Stochastic Gradient Descent (SGD). In the era of ever decreasing loss functions, SGD and its various…
math.OC2020★ 9 cited
On the Impossibility of Global Convergence in Multi-Loss Optimization
Alistair Letcher
Under mild regularity conditions, gradient-based methods converge globally to a critical point in the single-loss setting. This is known to break down for vanilla gradient descent…