Efficient Neural Network Training via Subset Pretraining

Jan Spörer, Bernhard Bermeitinger, Tomas Hrycej +2

In training neural networks, it is common practice to use partial gradients computed over batches, mostly very small subsets of the training set. This approach is motivated by the…