Tracing the Path to Grokking: Embeddings, Dropout, and Network Activation

Ahmed Salah, David Yevick

Grokking refers to delayed generalization in which the increase in test accuracy of a neural network occurs appreciably after the improvement in training accuracy This paper introd…