10 papers
Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives
Konstantinos Oikonomidis, Jan Quan, Kimon Antonakopoulos +3
In this work, we develop proximal preconditioned gradient methods with a focus on spectral gradient methods providing a proximal extension to the Muon and Scion optimizers. We intr…
On the Regularity of Generalized Conjugate Functions
Konstantinos Oikonomidis, Emanuel Laude, Panagiotis Patrinos
We investigate regularity properties of generalized conjugate functions induced by a general coupling function and the associated generalized proximal mapping. Our main results pro…
Nonlinearly preconditioned gradient flows
Konstantinos Oikonomidis, Alexander Bodard, Jan Quan +1
We study a continuous-time dynamical system which arises as the limit of a broad class of nonlinearly preconditioned gradient methods. Under mild assumptions, we establish existenc…
Scaled relative graphs for pairs of operators beyond classical monotonicity
Jan Quan, Alexander Bodard, Konstantinos Oikonomidis +1
We introduce a generalization of the scaled relative graph (SRG) to pairs of operators, enabling the visualization of their relative incremental properties. This novel SRG framewor…
Nonlinearly Preconditioned Gradient Methods: Momentum and Stochastic Analysis
Konstantinos Oikonomidis, Jan Quan, Panagiotis Patrinos
We study nonlinearly preconditioned gradient methods for smooth nonconvex optimization problems, focusing on sigmoid preconditioners that inherently perform a form of gradient clip…
Forward-backward splitting under the light of generalized convexity
Konstantinos Oikonomidis, Emanuel Laude, Panagiotis Patrinos
In this paper we present a unifying framework for continuous optimization methods grounded in the concept of generalized convexity. Utilizing the powerful theory of -convexity,…
Nonlinearly Preconditioned Gradient Methods under Generalized Smoothness
Konstantinos Oikonomidis, Jan Quan, Emanuel Laude +1
We analyze nonlinearly preconditioned gradient methods for solving smooth minimization problems. We introduce a generalized smoothness property, based on the notion of abstract con…
The inexact power augmented Lagrangian method for constrained nonconvex optimization
Alexander Bodard, Konstantinos Oikonomidis, Emanuel Laude +1
This work introduces an unconventional inexact augmented Lagrangian method where the augmenting term is a Euclidean norm raised to a power between one and two. The proposed algorit…
Adaptive proximal gradient methods are universal without approximation
Konstantinos A. Oikonomidis, Emanuel Laude, Puya Latafat +2
We show that adaptive proximal gradient methods for convex problems are not restricted to traditional Lipschitzian assumptions. Our analysis reveals that a class of linesearch-free…
Global Convergence Analysis of the Power Proximal Point and Augmented Lagrangian Method
Konstantinos A. Oikonomidis, Alexander Bodard, Emanuel Laude +1
In this paper we study an unconventional inexact Augmented Lagrangian Method (ALM) for convex optimization problems, as first proposed by Bertsekas, wherein the penalty term is a p…