A Mini-Block Fisher Method for Deep Neural Networks

Achraf Bahamou, Donald Goldfarb, Yi Ren

Deep neural networks (DNNs) are currently predominantly trained using first-order methods. Some of these methods (e.g., Adam, AdaGrad, and RMSprop, and their variants) incorporate…