Grokking as a Falsifiable Finite-Size Transition

Yuda Bi, Chenyu Zhang, Qiheng Wang +1

Grokking -- the delayed onset of generalization after early memorization -- is often described with phase-transition language, but that claim has lacked falsifiable finite-size inp…