Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networks

Jie Huang, Bruno Loureiro, Stefano Sarao Mannelli

We study the population loss landscape of two-layer ReLU networks of the form ${\sum }_{k=1}^K\mathrm{ReLU}(w_k^{\top }x)$ in a realisable teacher-student setting with Gaussian covaria…