Uncertainty-based Continual Learning with Adaptive Regularization

Abstract

We introduce a new neural network-based continual learning algorithm, dubbed as Uncertainty-regularized Continual Learning (UCL), which builds on traditional Bayesian online learning framework with variational inference. We focus on two significant drawbacks of the recently proposed regularization-based methods: a) considerable additional memory cost for determining the per-weight regularization strengths and b) the absence of gracefully forgetting scheme, which can prevent performance degradation in learning new tasks. In this paper, we show UCL can solve these two problems by introducing a fresh interpretation on the Kullback-Leibler (KL) divergence term of the variational lower bound for Gaussian mean-field approximation. Based on the interpretation, we propose the notion of node-wise uncertainty, which drastically reduces the number of additional parameters for implementing per-weight regularization. Moreover, we devise two additional regularization terms that enforce stability by freezing important parameters for past tasks and allow plasticity by controlling the actively learning parameters for a new task. Through extensive experiments, we show UCL convincingly outperforms most of recent state-of-the-art baselines not only on popular supervised learning benchmarks, but also on challenging lifelong reinforcement learning tasks. The source code of our algorithm is available at https://github.com/csm9493/UCL.

arXiv comment10 pages (including Supplementary Materials), Neurips 2019 camera ready version

Cited by in corpus (12)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• A continual learning survey: Defying forgetting in classification tasks
• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks
• CPR: Classifier-Projection Regularization for Continual Learning
• Preserving Earlier Knowledge in Continual Learning with the Help of All Previous Feature Extractors
• Gradual Fine-Tuning for Low-Resource Domain Adaptation
• Generalized Variational Continual Learning
• Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classification
• Learning Neural Models for Natural Language Processing in the Face of Distributional Shift
• TAG: Task-based Accumulated Gradients for Lifelong learning
• Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking