Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent

Mehdi Abbana Bennani, Thang Doan, Masashi Sugiyama

In Continual Learning settings, deep neural networks are prone to Catastrophic Forgetting. Orthogonal Gradient Descent was proposed to tackle the challenge. However, no theoretical…