FReLU: Flexible Rectified Linear Units for Improving Convolutional Neural Networks

Suo Qiu, Xiangmin Xu, Bolun Cai

Rectified linear unit (ReLU) is a widely used activation function for deep convolutional neural networks. However, because of the zero-hard rectification, ReLU networks miss the be…