571 citations · 729 across the 15 of their papers we have counts for
22 papers
A Trust-Region Algorithm for Noisy Equality Constrained Optimization
Shigeng Sun, Jorge Nocedal
This paper introduces a modified Byrd-Omojokun (BO) trust region algorithm to address the challenges posed by noisy function and gradient evaluations. The original BO method was de…
A Feasible Method for Constrained Derivative-Free Optimization
Melody Qiming Xuan, Jorge Nocedal
This paper explores a method for solving constrained optimization problems when the derivatives of the objective function are unavailable, while the derivatives of the constraints…
Design Guidelines for Noise-Tolerant Optimization with Applications in Robust Design
Yuchen Lou, Shigeng Sun, Jorge Nocedal
The development of nonlinear optimization algorithms capable of performing reliably in the presence of noise has garnered considerable attention lately. This paper advocates for st…
A Trust Region Method for the Optimization of Noisy Functions
Shigeng Sun, Jorge Nocedal
Classical trust region methods were designed to solve problems in which function and gradient information are exact. This paper considers the case when there are bounded errors (or…
Constrained Optimization in the Presence of Noise
Figen Oztoprak, Richard Byrd, Jorge Nocedal
The problem of interest is the minimization of a nonlinear function subject to nonlinear equality constraints using a sequential quadratic programming (SQP) method. The minimizatio…
Adaptive Finite-Difference Interval Estimation for Noisy Derivative-Free Optimization
Hao-Jun Michael Shi, Yuchen Xie, Melody Qiming Xuan +1
A common approach for minimizing a smooth nonlinear function is to employ finite-difference approximations to the gradient. While this can be easily performed when no error is pres…
On the Numerical Performance of Derivative-Free Optimization Methods Based on Finite-Difference Approximations
Hao-Jun Michael Shi, Melody Qiming Xuan, Figen Oztoprak +1
The goal of this paper is to investigate an approach for derivative-free optimization that has not received sufficient attention in the literature and is yet one of the simplest to…
Constrained and Composite Optimization via Adaptive Sampling Methods
Yuchen Xie, Raghu Bollapragada, Richard Byrd +1
The motivation for this paper stems from the desire to develop an adaptive sampling method for solving constrained optimization problems in which the objective function is stochast…
A Noise-Tolerant Quasi-Newton Algorithm for Unconstrained Optimization
Hao-Jun Michael Shi, Yuchen Xie, Richard Byrd +1
This paper describes an extension of the BFGS and L-BFGS methods for the minimization of a nonlinear function subject to errors. This work is motivated by applications that contain…
Analysis of the BFGS Method with Errors
Yuchen Xie, Richard Byrd, Jorge Nocedal
The classical convergence analysis of quasi-Newton methods assumes that the function and gradients employed at each iteration are exact. In this paper, we consider the case when th…
Derivative-Free Optimization of Noisy Functions via Quasi-Newton Methods
Albert S. Berahas, Richard H. Byrd, Jorge Nocedal
This paper presents a finite difference quasi-Newton method for the minimization of noisy functions. The method takes advantage of the scalability and power of BFGS updating, and e…
A Progressive Batching L-BFGS Method for Machine Learning
Raghu Bollapragada, Dheevatsa Mudigere, Jorge Nocedal +2
The standard L-BFGS method relies on gradient approximations that are not dominated by noise, so that search directions are descent directions, the line search is reliable, and qua…
Adaptive Sampling Strategies for Stochastic Optimization
Raghu Bollapragada, Richard Byrd, Jorge Nocedal
In this paper, we propose a stochastic optimization method that adaptively controls the sample size used in the computation of gradient approximations. Unlike other variance reduct…
An Investigation of Newton-Sketch and Subsampled Newton Methods
Albert S. Berahas, Raghu Bollapragada, Jorge Nocedal
Sketching, a dimensionality reduction technique, has received much attention in the statistics community. In this paper, we study sketching in the context of Newton's method for so…
Exact and Inexact Subsampled Newton Methods for Optimization
Raghu Bollapragada, Richard Byrd, Jorge Nocedal
The paper studies the solution of stochastic optimization problems in which approximations to the gradient and Hessian are obtained through subsampling. We first consider Newton-li…
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal +2
The stochastic gradient descent (SGD) method and its variants are algorithms of choice for many Deep Learning tasks. These methods operate in a small-batch regime wherein a fractio…
Optimization Methods for Large-Scale Machine Learning
Léon Bottou, Frank E. Curtis, Jorge Nocedal
This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case stud…
A Multi-Batch L-BFGS Method for Machine Learning
Albert S. Berahas, Jorge Nocedal, Martin Takáč
The question of how to parallelize the stochastic gradient descent (SGD) method has received much attention in the literature. In this paper, we focus instead on batch methods that…
A Second-Order Method for Convex -Regularized Optimization with Active Set Prediction
Nitish Shirish Keskar, Jorge Nocedal, Figen Oztoprak +1
We describe an active-set method for the minimization of an objective function that is the sum of a smooth convex function and an -regularization term. A distinctive fe…
An Algorithm for Quadratic -Regularized Optimization with a Flexible Active-Set Strategy
Stefan Solntsev, Jorge Nocedal, Richard Byrd
We present an active-set method for minimizing an objective that is the sum of a convex quadratic and regularization term. Unlike two-phase methods that combine a first-or…
A Stochastic Quasi-Newton Method for Large-Scale Optimization
R. H. Byrd, S. L. Hansen, J. Nocedal +1
The question of how to incorporate curvature information in stochastic approximation methods is challenging. The direct application of classical quasi- Newton updating techniques f…
An Inexact Successive Quadratic Approximation Method for Convex L-1 Regularized Optimization
Richard H. Byrd, Jorge Nocedal, Figen Oztoprak
We study a Newton-like method for the minimization of an objective function that is the sum of a smooth convex function and an l-1 regularization term. This method, which is someti…