Second-order Information Promotes Mini-Batch Robustness in Variance-Reduced Gradients

Sachin Garg, Albert S. Berahas, Michał Dereziński

We show that, for finite-sum minimization problems, incorporating partial second-order information of the objective function can dramatically improve the robustness to mini-batch s…