N-ReLU: Zero-Mean Stochastic Extension of ReLU

Md Motaleb Hossen Manik, Md Zabirul Islam, Ge Wang

Activation functions are fundamental for enabling nonlinear representations in deep neural networks. However, the standard rectified linear unit (ReLU) often suffers from inactive…