5 papers
Overfitting of Spectral Gradient Descent: How Matrix Geometry shapes Generalization and Implicit Bias
Guillaume Braun, Ichiro Hashimoto, Masaaki Imaizumi
We study the generalization of spectral gradient descent (SpecGD) in overparameterized matrix classification with corrupted labels. Each input combines a shared low-rank signal wit…
Neuron Block Dynamics for XOR Classification with Zero-Margin
Guillaume Braun, Masaaki Imaizumi
The ability of neural networks to learn useful features through stochastic gradient descent (SGD) is a cornerstone of their success. Most theoretical analyses focus on regression o…
Spectral Gradient Descent Mitigates Anisotropy-Driven Misalignment: A Case Study in Phase Retrieval
Guillaume Braun, Han Bao, Wei Huang +1
Spectral gradient methods, such as the Muon optimizer, modify gradient updates by preserving directional information while discarding scale, and have shown strong empirical perform…
Fast Escape, Slow Convergence: Learning Dynamics of Phase Retrieval under Power-Law Data
Guillaume Braun, Bruno Loureiro, Ha Quang Minh +1
Scaling laws describe how learning performance improves with data, compute, or training time, and have become a central theme in modern deep learning. We study this phenomenon in a…
Learning a Single Index Model from Anisotropic Data with vanilla Stochastic Gradient Descent
Guillaume Braun, Minh Ha Quang, Masaaki Imaizumi
We investigate the problem of learning a Single Index Model (SIM)- a popular model for studying the ability of neural networks to learn features - from anisotropic Gaussian inputs…