M-FAC: Efficient Matrix-Free Approximations of Second-Order Information

Elias Frantar, Eldar Kurtic, Dan Alistarh

Efficiently approximating local curvature information of the loss function is a key tool for optimization and compression of deep neural networks. Yet, most existing methods to app…