GraB: Finding Provably Better Data Permutations than Random Reshuffling

Yucheng Lu, Wentao Guo, Christopher De Sa

Random reshuffling, which randomly permutes the dataset each epoch, is widely adopted in model training because it yields faster convergence than with-replacement sampling. Recent…