Grokking in Linear Estimators -- A Solvable Model that Groks without Understanding

Noam Levi, Alon Beck, Yohai Bar-Sinai

Grokking is the intriguing phenomenon where a model learns to generalize long after it has fit the training data. We show both analytically and numerically that grokking can surpri…