On the Rate of Convergence of GD in Non-linear Neural Networks: An Adversarial Robustness Perspective

Guy Smorodinsky, Sveta Gimpleson, Itay Safran

We study the convergence dynamics of Gradient Descent (GD) in a minimal binary classification setting, consisting of a two-neuron ReLU network and two training instances. We prove…