2 citations · 2 across the 5 of their papers we have counts for
5 papers · 1 filter
The Implicit Bias of Depth: From Neural Collapse to Softmax Codes
Connall Garrod, Jonathan P. Keating, Christos Thrampoulidis
Neural collapse (NC) describes the structured geometry that emerges in the features and weights of trained classifiers. Recent theory suggests NC can be suboptimal in deep architec…
Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data
Chen Fan, Mark Schmidt, Christos Thrampoulidis
Different gradient-based methods for optimizing overparameterized models can all achieve zero training error yet converge to distinctly different solutions inducing different gener…
BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimization
Chen Fan, Gaspard Choné-Ducasse, Mark Schmidt +1
The popularity of bi-level optimization (BO) in deep learning has spurred a growing interest in studying gradient-based BO algorithms. However, existing algorithms involve two coup…
Fast Convergence of Random Reshuffling under Over-Parameterization and the Polyak-Łojasiewicz Condition
Chen Fan, Christos Thrampoulidis, Mark Schmidt
Modern machine learning models are often over-parameterized and as a result they can interpolate the training data. Under such a scenario, we study the convergence properties of a…
Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently
Haoyuan Sun, Kwangjun Ahn, Christos Thrampoulidis +1
Driven by the empirical success and wide use of deep neural networks, understanding the generalization performance of overparameterized models has become an increasingly popular qu…