Nondeterminism and Instability in Neural Network Optimization

Cecilia Summers, Michael J. Dinneen

Nondeterminism in neural network optimization produces uncertainty in performance, making small improvements difficult to discern from run-to-run variability. While uncertainty can…