Exploring Grokking: Experimental and Mechanistic Investigations

Hu Qiye, Zhou Hao, Yu RuoXi

The phenomenon of grokking in over-parameterized neural networks has garnered significant interest. It involves the neural network initially memorizing the training set with zero t…