Canalization Before Generalization: Grokking as a Dynamical Probe

Yiming Lin

For overparameterized neural networks, many solutions can fit the training data equally well while behaving very differently on unseen samples. Grokking separates training fit from…