On the Variance of Neural Network Training with respect to Test Sets and Distributions

Keller Jordan

Typical neural network trainings have substantial variance in test-set performance between repeated runs, impeding hyperparameter comparison and training reproducibility. In this w…