Neural Networks Remember More: The Power of Parameter Isolation and Combination

Biqing Zeng, Zehan Li, Aladdin Ayesh

Catastrophic forgetting is a pervasive issue for pre-trained language models (PLMs) during continual learning, where models lose previously acquired knowledge when sequentially tra…