Competition-based Adaptive ReLU for Deep Neural Networks

Junjia Chen, Zhibin Pan

Activation functions introduce nonlinearity into deep neural networks. Most popular activation functions allow positive values to pass through while blocking or suppressing negativ…