Subspace Crystallization in Grokking: Rank Compression Tracks the Phase Transition

Shlok Mehendale, Paras Chopra

Grokking describes understanding something deeply and intuitively. In neural networks, it is the phenomenon where a network generalizes long after memorizing its training data. Thi…