22 citations · 38 across the 9 of their papers we have counts for
4 papers · 1 filter
Stochastic Adaptive Gradient Descent Without Descent
Jean-François Aujol, Jérémie Bigot, Camille Castera
We introduce a new adaptive step-size strategy for convex optimization with stochastic gradient that exploits the local geometry of the objective function only by means of a first-…
From Learning to Optimize to Learning Optimization Algorithms
Camille Castera, Peter Ochs
Towards designing learned optimization algorithms that are usable beyond their training setting, we identify key principles that classical algorithms obey, but have up to now, not…
Second-order step-size tuning of SGD for non-convex optimization
Camille Castera, Jérôme Bolte, Cédric Févotte +1
In view of a direct and simple improvement of vanilla SGD, this paper presents a fine-tuning of its step-sizes in the mini-batch case. For doing so, one estimates curvature, based…
An Inertial Newton Algorithm for Deep Learning
Camille Castera, Jérôme Bolte, Cédric Févotte +1
We introduce a new second-order inertial optimization method for machine learning called INNA. It exploits the geometry of the loss function while only requiring stochastic approxi…