2 papers
cs.LG2025
Nonlinearity Enhanced Adaptive Activation Functions
David Yevick
A general procedure for introducing parametric, learned, nonlinearity into activation functions is found to enhance the accuracy of representative neural networks without requiring…
cs.LG2024
Controlling Grokking with Nonlinearity and Data Symmetry
Ahmed Salah, David Yevick
This paper demonstrates that grokking behavior in modular arithmetic with a modulus P in a neural network can be controlled by modifying the profile of the activation function as w…