7 papers
PANOC-lite: A simpler and more efficient algorithm for composite minimization
Alexander Bodard, Pieter Pas, Andreas Themelis +1
This work introduces a simple and efficient linesearch method for composite minimization that accelerates proximal-gradient iterations with fast Newton-type directions. Our algorit…
On the Regularity of Generalized Conjugate Functions
Konstantinos Oikonomidis, Emanuel Laude, Panagiotis Patrinos
We investigate regularity properties of generalized conjugate functions induced by a general coupling function and the associated generalized proximal mapping. Our main results pro…
Nonlinearly preconditioned gradient flows
Konstantinos Oikonomidis, Alexander Bodard, Jan Quan +1
We study a continuous-time dynamical system which arises as the limit of a broad class of nonlinearly preconditioned gradient methods. Under mild assumptions, we establish existenc…
Rethinking PCA Through Duality
Jan Quan, Johan Suykens, Panagiotis Patrinos
Motivated by the recently shown connection between self-attention and (kernel) principal component analysis (PCA), we revisit the fundamentals of PCA. Using the difference-of-conve…
Nonlinearly Preconditioned Gradient Methods: Momentum and Stochastic Analysis
Konstantinos Oikonomidis, Jan Quan, Panagiotis Patrinos
We study nonlinearly preconditioned gradient methods for smooth nonconvex optimization problems, focusing on sigmoid preconditioners that inherently perform a form of gradient clip…
Escaping saddle points without Lipschitz smoothness: the power of nonlinear preconditioning
Alexander Bodard, Panagiotis Patrinos
We study generalized smoothness in nonconvex optimization, focusing on -smoothness and anisotropic smoothness. The former was empirically derived from practical neural…