SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structures

Julian Kranz, Davide Gallon, Steffen Dereich +1

We study gradient flows for loss landscapes of fully connected feedforward neural networks with commonly used continuously differentiable activation functions such as the logistic,…