Distributional Spectral Diagnostics for Localizing Grokking Transitions

Ziyue Wang, Yufeng Ying, Takafumi Kanamori

In grokking, a model first fits the training data while test accuracy remains low, and only later begins to generalize. We ask whether this transition can be localized from observe…