Does a sparse ReLU network training problem always admit an optimum?

Quoc-Tung Le, Elisa Riccietti, Rémi Gribonval

Given a training set, a loss function, and a neural network architecture, it is often taken for granted that optimal network parameters exist, and a common practice is to apply ava…