Controlling Grokking with Nonlinearity and Data Symmetry

Ahmed Salah, David Yevick

This paper demonstrates that grokking behavior in modular arithmetic with a modulus P in a neural network can be controlled by modifying the profile of the activation function as w…