2 citations · 2 across the 5 of their papers we have counts for
5 papers · 1 filter
Nonlinearly preconditioned gradient flows
Konstantinos Oikonomidis, Alexander Bodard, Jan Quan +1
We study a continuous-time dynamical system which arises as the limit of a broad class of nonlinearly preconditioned gradient methods. Under mild assumptions, we establish existenc…
Scaled relative graphs for pairs of operators beyond classical monotonicity
Jan Quan, Alexander Bodard, Konstantinos Oikonomidis +1
We introduce a generalization of the scaled relative graph (SRG) to pairs of operators, enabling the visualization of their relative incremental properties. This novel SRG framewor…
Rethinking PCA Through Duality
Jan Quan, Johan Suykens, Panagiotis Patrinos
Motivated by the recently shown connection between self-attention and (kernel) principal component analysis (PCA), we revisit the fundamentals of PCA. Using the difference-of-conve…
Nonlinearly Preconditioned Gradient Methods: Momentum and Stochastic Analysis
Konstantinos Oikonomidis, Jan Quan, Panagiotis Patrinos
We study nonlinearly preconditioned gradient methods for smooth nonconvex optimization problems, focusing on sigmoid preconditioners that inherently perform a form of gradient clip…
Nonlinearly Preconditioned Gradient Methods under Generalized Smoothness
Konstantinos Oikonomidis, Jan Quan, Emanuel Laude +1
We analyze nonlinearly preconditioned gradient methods for solving smooth minimization problems. We introduce a generalized smoothness property, based on the notion of abstract con…