Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint

Dong Yin, Mehrdad Farajtabar, Ang Li +2

Neural networks have achieved remarkable success in many cognitive tasks. However, when they are trained sequentially on multiple tasks without access to old data, their performanc…