Large batch size training of neural networks with adversarial training and second-order information

Zhewei Yao, Amir Gholami, Daiyaan Arfeen +4

The most straightforward method to accelerate Stochastic Gradient Descent (SGD) computation is to distribute the randomly selected batch of inputs over multiple processors. To keep…