10 papers
Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives
Konstantinos Oikonomidis, Jan Quan, Kimon Antonakopoulos +3
In this work, we develop proximal preconditioned gradient methods with a focus on spectral gradient methods providing a proximal extension to the Muon and Scion optimizers. We intr…
Nonlinearly preconditioned gradient flows
Konstantinos Oikonomidis, Alexander Bodard, Jan Quan +1
We study a continuous-time dynamical system which arises as the limit of a broad class of nonlinearly preconditioned gradient methods. Under mild assumptions, we establish existenc…
On the Regularity of Generalized Conjugate Functions
Konstantinos Oikonomidis, Emanuel Laude, Panagiotis Patrinos
We investigate regularity properties of generalized conjugate functions induced by a general coupling function and the associated generalized proximal mapping. Our main results pro…
Scaled relative graphs for pairs of operators beyond classical monotonicity
Jan Quan, Alexander Bodard, Konstantinos Oikonomidis +1
We introduce a generalization of the scaled relative graph (SRG) to pairs of operators, enabling the visualization of their relative incremental properties. This novel SRG framewor…
The inexact power augmented Lagrangian method for constrained nonconvex optimization
Alexander Bodard, Konstantinos Oikonomidis, Emanuel Laude +1
This work introduces an unconventional inexact augmented Lagrangian method where the augmenting term is a Euclidean norm raised to a power between one and two. The proposed algorit…
Nonlinearly Preconditioned Gradient Methods: Momentum and Stochastic Analysis
Konstantinos Oikonomidis, Jan Quan, Panagiotis Patrinos
We study nonlinearly preconditioned gradient methods for smooth nonconvex optimization problems, focusing on sigmoid preconditioners that inherently perform a form of gradient clip…