Reconstructing Deep Neural Networks: Unleashing the Optimization Potential of Natural Gradient Descent

Weihua Liu, Said Boumaraf, Jianwu Li +4

Natural gradient descent (NGD) is a powerful optimization technique for machine learning, but the computational complexity of the inverse Fisher information matrix limits its appli…