Overparametrization bends the landscape: BBP transitions at initialization in simple Neural Networks

Brandon Livio Annesi, Dario Bocchi, Chiara Cammarota

High-dimensional non-convex loss landscapes play a central role in the theory of Machine Learning. Gaining insight into how these landscapes interact with gradient-based optimizati…