2 papers
cs.LG2023
Analysis and Comparison of Two-Level KFAC Methods for Training Deep Neural Networks
Abdoulaye Koroko, Ani Anciaux-Sedrakian, Ibtihel Ben Gharbia +3
As a second-order method, the Natural Gradient Descent (NGD) has the ability to accelerate training of neural networks. However, due to the prohibitive computational and memory cos…
cs.NE2022
Efficient Approximations of the Fisher Matrix in Neural Networks using Kronecker Product Singular Value Decomposition
Abdoulaye Koroko, Ani Anciaux-Sedrakian, Ibtihel Ben Gharbia +3
Several studies have shown the ability of natural gradient descent to minimize the objective function more efficiently than ordinary gradient descent based methods. However, the bo…