Self-Preservation in Language Models

Dongkyu Cho, Rumi Chunara, Ali Behrouz

Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while f…