Efficient Approximations of the Fisher Matrix in Neural Networks using Kronecker Product Singular Value Decomposition

Abdoulaye Koroko, Ani Anciaux-Sedrakian, Ibtihel Ben Gharbia +3

Several studies have shown the ability of natural gradient descent to minimize the objective function more efficiently than ordinary gradient descent based methods. However, the bo…