Grokking in Linear Models for Logistic Regression

Nataraj Das, Atreya Vedantam, Chandrashekar Lakshminarayanan

Grokking, the phenomenon of delayed generalization, is often attributed to the depth and compositional structure of deep neural networks. We study grokking in one of the simplest p…