Learning DNN networks using un-rectifying ReLU with compressed sensing application

Wen-Liang Hwang, Shih-Shuo Tung

The un-rectifying technique expresses a non-linear point-wise activation function as a data-dependent variable, which means that the activation variable along with its input and ou…