Overfitting of Spectral Gradient Descent: How Matrix Geometry shapes Generalization and Implicit Bias

Guillaume Braun, Ichiro Hashimoto, Masaaki Imaizumi

We study the generalization of spectral gradient descent (SpecGD) in overparameterized matrix classification with corrupted labels. Each input combines a shared low-rank signal wit…