Topological Signatures of Grokking

Yifan Tang, Qiquan Wang, Inés García-Redondo +1

We study the grokking phenomenon through the lens of topology. Using persistent homology on point clouds derived from the embedding matrices of a range of models trained on modular…