571 citations · 729 across the 15 of their papers we have counts for
4 papers · 1 filter
Exact and Inexact Subsampled Newton Methods for Optimization
Raghu Bollapragada, Richard Byrd, Jorge Nocedal
The paper studies the solution of stochastic optimization problems in which approximations to the gradient and Hessian are obtained through subsampling. We first consider Newton-li…
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal +2
The stochastic gradient descent (SGD) method and its variants are algorithms of choice for many Deep Learning tasks. These methods operate in a small-batch regime wherein a fractio…
Optimization Methods for Large-Scale Machine Learning
Léon Bottou, Frank E. Curtis, Jorge Nocedal
This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case stud…
A Multi-Batch L-BFGS Method for Machine Learning
Albert S. Berahas, Jorge Nocedal, Martin Takáč
The question of how to parallelize the stochastic gradient descent (SGD) method has received much attention in the literature. In this paper, we focus instead on batch methods that…