Grokking vs. Learning: Same Features, Different Encodings

Dmitry Manning-Coe, Jacopo Gliozzi, Alexander G. Stapleton +4

Grokking typically achieves similar loss to ordinary, "steady", learning. We ask whether these different learning paths - grokking versus ordinary training - lead to fundamental di…