53 citations · 183 across the 18 of their papers we have counts for
5 papers · 1 filter
Sub-network Discovery and Soft-masking for Continual Learning of Mixed Tasks
Zixuan Ke, Bing Liu, Wenhan Xiong +2
Continual learning (CL) has two main objectives: preventing catastrophic forgetting (CF) and encouraging knowledge transfer (KT). The existing literature mainly focused on overcomi…
Parameter-Level Soft-Masking for Continual Learning
Tatsuya Konishi, Mori Kurokawa, Chihiro Ono +3
Existing research on task incremental learning in continual learning has primarily focused on preventing catastrophic forgetting (CF). Although several techniques have achieved lea…
Open-World Continual Learning: Unifying Novelty Detection and Continual Learning
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi +2
As AI agents are increasingly used in the real open world with unknowns or novelties, they need the ability to (1) recognize objects that (a) they have learned before and (b) detec…
Continual Pre-training of Language Models
Zixuan Ke, Yijia Shao, Haowei Lin +3
Language models (LMs) have been instrumental for the rapid advance of natural language processing. This paper studies continual pre-training of LMs, in particular, continual domain…
Adapting a Language Model While Preserving its General Knowledge
Zixuan Ke, Yijia Shao, Haowei Lin +3
Domain-adaptive pre-training (or DA-training for short), also known as post-training, aims to train a pre-trained general-purpose language model (LM) using an unlabeled corpus of a…