Rethink ReLU to Training Better CNNs

Gangming Zhao, Zhaoxiang Zhang, He Guan +2

Most of convolutional neural networks share the same characteristic: each convolutional layer is followed by a nonlinear activation layer where Rectified Linear Unit (ReLU) is the…