Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape

Kedar Karhadkar, Michael Murray, Hanna Tseran +1

We study the loss landscape of both shallow and deep, mildly overparameterized ReLU neural networks on a generic finite input dataset for the squared error loss. We show both by co…