Nonparametric Learning of Two-Layer ReLU Residual Units

Zhunxuan Wang, Linyun He, Chunchuan Lyu +1

We describe an algorithm that learns two-layer residual units using rectified linear unit (ReLU) activation: suppose the input $\mathbf{x}$ is from a distribution with support spac…