most citedDichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking

2 citations · 2 across the 1 of their papers we have counts for

collaborators

1 paper

cs.LG2023★ 2 cited

Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking

Kaifeng Lyu, Jikai Jin, Zhiyuan Li +3

Recent work by Power et al. (2022) highlighted a surprising "grokking" phenomenon in learning arithmetic tasks: a neural net first "memorizes" the training set, resulting in perfec…