Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks

David A. R. Robin, Kevin Scaman, Marc Lelarge

We present a framework to define a large class of neural networks for which, by construction, training by gradient flow provably reaches arbitrarily low loss when the number of par…