How to explain grokking
arXiv:2412.18624
Abstract
Explanation of grokking (delayed generalization) in learning is given by modeling grokking by the stochastic gradient Langevin dynamics (Brownian motion) and applying the ideas of thermodynamics.
arXiv commentDiscussion slightly improved