Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learning

Ishir Garg, Neel Kolhe, Andy Peng +1

Continual learning aims to enable neural networks to acquire new knowledge on sequential tasks. However, the key challenge in such settings is to learn new tasks without catastroph…