Speeding up Convolutional Neural Networks By Exploiting the Sparsity of Rectifier Units

Shaohuai Shi, Xiaowen Chu

Rectifier neuron units (ReLUs) have been widely used in deep convolutional networks. An ReLU converts negative values to zeros, and does not change positive values, which leads to…