TeLU Activation Function for Fast and Stable Deep Learning

Alfredo Fernandez, Ankur Mali

We propose the Hyperbolic Tangent Exponential Linear Unit (TeLU), a neural network hidden activation function defined as TeLU(x)=xtanh(exp(x)). TeLU's design is grounded in the cor…