6 citations · 14 across the 20 of their papers we have counts for
22 papers
Stochastic convergence of parallel asynchronous adaptive first-order methods
Serge Gratton, Philippe L. Toint
A new class of asynchronous adaptive first-order optimization methods is introduced, comprising asynchronous variants of several popular algorithms. Versions of these methods using…
A unified convergence theory for adaptive first-order methods in the nonconvex case, including AdaNorm, full and diagonal AdaGrad and Muon
S. Gratton, Ph. L. Toint
A unified framework for first-order optimization algorithms fornonconvex unconstrained optimization is proposed that uses adaptivelypreconditioned gradients and includes popular me…
An objective-function-free algorithm for nonconvex stochastic optimization with deterministic equality and inequality constraints
S. Gratton, Ph. L. Toint
An algorithm is proposed for solving optimization problems with stochastic objective and deterministic equality and inequality constraints. This algorithm is objective-function-fre…
Iteration complexity of the Difference-of-Convex Algorithm for unconstrained optimization: a simple proof
Serge Gratton, Philippe L. Toint
We propose a simple proof of the worst-case iteration complexity for the Difference of Convex functions Algorithm (DCA) for unconstrained minimization, showing that the global rate…
ADSWITCH: A Simple First-Order Algorithm for Full-Rank Equality Constrained Optimization
Serge Gratton, Philippe L. Toint
A very simple first-order algorithm is proposed for solving nonlinear optimization problems with deterministic nonlinear equality constraints. This algorithm adaptively selects ste…
Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimization
Serge Gratton, Alena Kopaničáková, Philippe Toint
Two OFFO (Objective-Function Free Optimization) noise tolerant algorithms are presented that handle bound constraints, inexact gradients and use second-order information when avail…
Fast Stochastic Second-Order Adagrad for Nonconvex Bound-Constrained Optimization
S. Bellavia, S. Gratton, B. Morini +1
ADAGB2, a generalization of the Adagrad algorithm for stochastic optimization is introduced, which is also applicable to bound-constrained problems and capable of using second-orde…
A Fast Newton Method Under Local Lipschitz Smoothness
Serge Gratton, Sadok Jerad, Philippe L. Toint
A new, fast second-order method is proposed that achieves the optimal complexity to obtain first-order -stationary points. Crucially,…
prunAdag: an adaptive pruning-aware gradient method
Margherita Porcelli, Giovanni Seraghiti, Philippe L. Toint
A pruning-aware adaptive gradient method is proposed which classifies the variables in two sets before updating them using different strategies. This technique extends the ``releva…
Examples of slow convergence for adaptive regularization optimization methods are not isolated
Philippe L. Toint
The adaptive regularization algorithm for unconstrained nonconvex optimization was shown in Nesterov and Polyak (2006) and Cartis, Gould and Toint (2011) to require, under standard…
Refining asymptotic complexity bounds for nonconvex optimization methods, including why steepest descent is rather than
Serge Gratton, Chee-Khian Sim, Philippe L. Toint
We revisit the standard ``telescoping sum'' argument ubiquitous in the final steps of analyzing evaluation complexity of algorithms for smooth nonconvex optimization, and obtain a…
S2MPJ and CUTEst optimization problems for Matlab, Python and Julia
Serge Gratton, Philippe L. Toint
A new decoder for the SIF test problems of the CUTEst collection is described, which produces problem files allowing the computation of values and derivatives of the objective func…
An optimally fast objective-function-free minimization algorithm using random subspaces
S. Bellavia, S. Gratton, B. Morini +1
An algorithm for unconstrained non-convex optimization is described, which does not evaluate the objective function and in which minimization is carried out, at each iteration, wit…
Divergence of the ADAM algorithm with fixed-stepsize: a (very) simple example
Ph. L. Toint
A very simple unidimensional function with Lipschitz continuous gradient is constructed such that the ADAM algorithm with constant stepsize, started from the origin, diverges when…
OFFO minimization algorithms for second-order optimality and their complexity
S. Gratton, Ph. L. Toint
An Adagrad-inspired class of algorithms for smooth unconstrained optimization is presented in which the objective function is never evaluated and yet the gradient norms decrease at…
Complexity and performance for two classes of noise-tolerant first-order algorithms
S. Gratton, S. Jerad, Ph. L. Toint
Two classes of algorithms for optimization in the presence of noise are presented, that do not require the evaluation of the objective function. The first generalizes the well-know…
Complexity of a Class of First-Order Objective-Function-Free Optimization Algorithms
S. Gratton, S. Jerad, Ph. L. Toint
A parametric class of trust-region algorithms for unconstrained nonconvex optimization is considered where the value of the objective function is never computed. The class contains…
Trust-region algorithms: probabilistic complexity and intrinsic noise with applications to subsampling techniques
S. Bellavia, G. Gurioli, B. Morini +1
A trust-region algorithm is presented for finding approximate minimizers of smooth unconstrained functions whose values and derivatives are subject to random noise. It is shown tha…
Hölder Gradient Descent and Adaptive Regularization Methods in Banach Spaces for First-Order Points
Serge Gratton, Sadok Jerad, Philippe L. Toint
This paper considers optimization of smooth nonconvex functionals in smooth infinite dimensional spaces. A Hölder gradient descent algorithm is first proposed for finding approxima…
A note on solving nonlinear optimization problems in variable precision
S. Gratton, Ph. L. Toint
This short note considers an efficient variant of the trust-region algorithm with dynamic accuracy proposed Carter (1993) and Conn, Gould and Toint (2000) as a tool for very high-p…
Minimizing convex quadratic with variable precision conjugate gradients
S. Gratton, E. Simon, D. Titley-Peloquin +1
We investigate the method of conjugate gradients, exploiting inaccurate matrix-vector products, for the solution of convex quadratic optimization problems. Theoretical performance…
A note on preconditioning weighted linear least squares, with consequences for weakly-constrained variational data assimilation
Serge Gratton, Selime Gürol, Ehouarn Simon +1
The effect of preconditioning linear weighted least-squares using an approximation of the model matrix is analyzed, showing the interplay of the eigenstructures of both the model a…