Mish: A Self Regularized Non-Monotonic Activation Function

Diganta Misra

We propose $\text{Mish}$, a novel self-regularized non-monotonic activation function which can be mathematically defined as: $f(x)=x\tanh (softplus(x))$. As activation functions p…