6 papers · 1 filter
Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives
Konstantinos Oikonomidis, Jan Quan, Kimon Antonakopoulos +3
In this work, we develop proximal preconditioned gradient methods with a focus on spectral gradient methods providing a proximal extension to the Muon and Scion optimizers. We intr…
Nonlinearly preconditioned gradient flows
Konstantinos Oikonomidis, Alexander Bodard, Jan Quan +1
We study a continuous-time dynamical system which arises as the limit of a broad class of nonlinearly preconditioned gradient methods. Under mild assumptions, we establish existenc…
Scaled relative graphs for pairs of operators beyond classical monotonicity
Jan Quan, Alexander Bodard, Konstantinos Oikonomidis +1
We introduce a generalization of the scaled relative graph (SRG) to pairs of operators, enabling the visualization of their relative incremental properties. This novel SRG framewor…
Scaled Relative Graphs for Nonmonotone Operators with Applications in Circuit Theory
Jan Quan, Brecht Evens, Rodolphe Sepulchre +1
The scaled relative graph (SRG) is a powerful graphical tool for analyzing the properties of operators, by mapping their graph onto the complex plane. In this work, we study the SR…
Nonlinearly Preconditioned Gradient Methods: Momentum and Stochastic Analysis
Konstantinos Oikonomidis, Jan Quan, Panagiotis Patrinos
We study nonlinearly preconditioned gradient methods for smooth nonconvex optimization problems, focusing on sigmoid preconditioners that inherently perform a form of gradient clip…
Nonlinearly Preconditioned Gradient Methods under Generalized Smoothness
Konstantinos Oikonomidis, Jan Quan, Emanuel Laude +1
We analyze nonlinearly preconditioned gradient methods for solving smooth minimization problems. We introduce a generalized smoothness property, based on the notion of abstract con…