Gradient Regularized Natural Gradients

Satya Prakash Dash, Hossein Abdi, Wei Pan +2

Gradient regularization (GR) has been shown to improve the generalizability of trained models. While Natural Gradient Descent has been shown to accelerate optimization in the initi…