13 citations · 13 across the 2 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2019★ 13 cited
The Role of Memory in Stochastic Optimization
Antonio Orvieto, Jonas Kohler, Aurelien Lucchi
The choice of how to retain information about past gradients dramatically affects the convergence properties of state-of-the-art stochastic optimization methods, such as Heavy-ball…
cs.LG2019
Adaptive norms for deep learning with regularized Newton methods
Jonas Kohler, Leonard Adolphs, Aurelien Lucchi
We investigate the use of regularized Newton methods with adaptive norms for optimizing neural networks. This approach can be seen as a second-order counterpart of adaptive gradien…