4 papers
SGD at the Edge of Stability: Stochastic Stabilization with Large Learning Rates
Konstantinos Emmanouilidis, Lachlan MacDonald, Salma Tarmoun +1
Modern deep learning has been shown to operate at the edge of stability, routinely using learning rates far larger than those justified by classical optimization theory. Most prior…
Certified Robustness from Approximate Gaussian Mixture Structures in Pretrained Latent Spaces
Konstantinos Emmanouilidis, Tianjiao Ding, Nghia Nguyen +2
Deep learning models are vulnerable to adversarial perturbations, raising important concerns for safety-critical deployment. Empirical defenses can achieve strong robustness in pra…
Shuffling the Data, Stretching the Step-size: Sharper Bias in constant step-size SGD
Konstantinos Emmanouilidis, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Rene Vidal
From adversarial robustness to multi-agent learning, many machine learning tasks can be cast as finite-sum min-max optimization or, more generally, as variational inequality proble…
Stochastic Extragradient with Random Reshuffling: Improved Convergence for Variational Inequalities
Konstantinos Emmanouilidis, René Vidal, Nicolas Loizou
The Stochastic Extragradient (SEG) method is one of the most popular algorithms for solving finite-sum min-max optimization and variational inequality problems (VIPs) appearing in…