A Kronecker-factored approximate Fisher matrix for convolution layers

Roger Grosse, James Martens

Second-order optimization methods such as natural gradient descent have the potential to speed up training of neural networks by correcting for the curvature of the loss function.…