GD doesn't make the cut: Three ways that non-differentiability affects neural network training

Siddharth Krishna Kumar

This paper critically examines the fundamental distinctions between gradient methods applied to non-differentiable functions (NGDMs) and classical gradient descents (GDs) for diffe…