Deriving Activation Functions Using Integration

Allen Hao Huang, Imanol Schlag

Our work proposes a novel approach to designing activation functions by focusing on their gradients and deriving the corresponding activation functions using integration. We introd…