Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization

Gang Wang, Georgios B. Giannakis, Jie Chen

Neural networks with REctified Linear Unit (ReLU) activation functions (a.k.a. ReLU networks) have achieved great empirical success in various domains. Nonetheless, existing result…