Vision Transformers that Never Stop Learning

Caihao Sun, Mingqi Yuan, Shiyuan Wang +1

Loss of plasticity refers to the progressive inability of a model to adapt to new tasks and poses a fundamental challenge for continual learning. While this phenomenon has been ext…