Mapping the Emergence of Regularization-Driven Dynamics in Grokking

Yiming Lin, Yuxuan Wang

For overparameterized neural networks, many solutions can fit the training data equally well while behaving very differently on unseen samples. Grokking separates training fit from…