In almost all shallow analytic neural network optimization landscapes, efficient minimizers have strongly convex neighborhoods

Felix Benning, Steffen Dereich

Whether or not a local minimum of a cost function has a strongly convex neighborhood greatly influences the asymptotic convergence rate of optimizers. In this article, we rigorousl…