3.5k citations
- University of California, BerkeleyUS107 papers
- Berkeley CollegeUS13 papers
- Lawrence Berkeley National LaboratoryUS11 papers
- The University of QueenslandAU10 papers
- Centre National de la Recherche ScientifiqueFR9 papers
- Stanford UniversityUS9 papers
- Max Planck Institute for InformaticsDE8 papers
- Université Paris-SaclayFR8 papers
- Goddard Space Flight CenterUS7 papers
- Institut d'Astrophysique SpatialeFR7 papers
- University of California San DiegoUS7 papers
- University of ManchesterGB7 papers
7 papers · 1 filter
DINGO: Distributed Newton-Type Method for Gradient-Norm Optimization
Rixon Crane, Fred Roosta
For optimization of a sum of functions in a distributed computing environment, we present a novel communication efficient Newton-type algorithm that enjoys a variety of advantages…
Newton-MR: Inexact Newton Method With Minimum Residual Sub-problem Solver
Fred Roosta, Yang Liu, Peng Xu +1
We consider a variant of inexact Newton Method, called Newton-MR, in which the least-squares sub-problems are solved approximately using Minimum Residual method. By construction, N…
Inexact Non-Convex Newton-Type Methods
Zhewei Yao, Peng Xu, Farbod Roosta-Khorasani +1
For solving large-scale non-convex problems, we propose inexact variants of trust region and adaptive cubic regularization methods, which, to increase efficiency, incorporate vario…
Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study
Peng Xu, Farbod Roosta-Khorasani, Michael W. Mahoney
While first-order optimization methods such as stochastic gradient descent (SGD) are popular in machine learning (ML), they come with well-known deficiencies, including relatively-…
Newton-Type Methods for Non-Convex Optimization Under Inexact Hessian Information
Peng Xu, Fred Roosta, Michael W. Mahoney
We consider variants of trust-region and cubic regularization methods for non-convex optimization, in which the Hessian matrix is approximated. Under mild conditions on the inexact…
FLAG n' FLARE: Fast Linearly-Coupled Adaptive Gradient Methods
Xiang Cheng, Farbod Roosta-Khorasani, Stefan Palombo +2
We consider first order gradient methods for effectively optimizing a composite objective in the form of a sum of smooth and, potentially, non-smooth functions. We present accelera…