4 papers
Rethinking PCA Through Duality
Jan Quan, Johan Suykens, Panagiotis Patrinos
Motivated by the recently shown connection between self-attention and (kernel) principal component analysis (PCA), we revisit the fundamentals of PCA. Using the difference-of-conve…
Nonlinearly Preconditioned Gradient Methods: Momentum and Stochastic Analysis
Konstantinos Oikonomidis, Jan Quan, Panagiotis Patrinos
We study nonlinearly preconditioned gradient methods for smooth nonconvex optimization problems, focusing on sigmoid preconditioners that inherently perform a form of gradient clip…
Nonlinearly Preconditioned Gradient Methods under Generalized Smoothness
Konstantinos Oikonomidis, Jan Quan, Emanuel Laude +1
We analyze nonlinearly preconditioned gradient methods for solving smooth minimization problems. We introduce a generalized smoothness property, based on the notion of abstract con…
Scaled Relative Graphs for Nonmonotone Operators with Applications in Circuit Theory
Jan Quan, Brecht Evens, Rodolphe Sepulchre +1
The scaled relative graph (SRG) is a powerful graphical tool for analyzing the properties of operators, by mapping their graph onto the complex plane. In this work, we study the SR…