Provable Training of a ReLU Gate with an Iterative Non-Gradient Algorithm

Sayar Karmakar, Anirbit Mukherjee

In this work, we demonstrate provable guarantees on the training of a single ReLU gate in hitherto unexplored regimes. We give a simple iterative stochastic algorithm that can trai…