44 citations · 54 across the 16 of their papers we have counts for
7 papers · 1 filter
Primal-Dual Inexact Newton-MR for Nonconvex Optimization with Equality Constraints
Oscar Smee, Fred Roosta
Optimization problems with nonlinear equality constraints arise throughout science, engineering, and increasingly in machine learning. Prominent methods for solving such problems i…
First-ish Order Methods: Hessian-aware Scalings of Gradient Descent
Oscar Smee, Fred Roosta, Stephen J. Wright
Gradient descent is the primary workhorse for optimizing large-scale problems in machine learning. However, its performance is highly sensitive to the choice of the learning rate.…
Inexact Newton-type Methods for Optimisation with Nonnegativity Constraints
Oscar Smee, Fred Roosta
We consider solving large scale nonconvex optimisation problems with nonnegativity constraints. Such problems arise frequently in machine learning, such as nonnegative least-square…
Non-Uniform Smoothness for Gradient Descent
Albert S. Berahas, Lindon Roberts, Fred Roosta
The analysis of gradient descent-type methods typically relies on the Lipschitz continuity of the objective gradient. This generally requires an expensive hyperparameter tuning pro…
Complexity Guarantees for Nonconvex Newton-MR Under Inexact Hessian Information
Alexander Lim, Fred Roosta
We consider an extension of the Newton-MR algorithm for nonconvex unconstrained optimization to the settings where Hessian information is approximated. Under a particular noise mod…
DINO: Distributed Newton-Type Optimization Method
Rixon Crane, Fred Roosta
We present a novel communication-efficient Newton-type algorithm for finite-sum optimization over a distributed computing environment. Our method, named DINO, overcomes both theore…