EraseReLU: A Simple Way to Ease the Training of Deep Convolution Neural Networks

Xuanyi Dong, Guoliang Kang, Kun Zhan +1

For most state-of-the-art architectures, Rectified Linear Unit (ReLU) becomes a standard component accompanied with each layer. Although ReLU can ease the network training to an ex…