Spectral Collapse Drives Loss of Plasticity in Deep Continual Learning

Arjun Prakash, Naicheng He, Kaicheng Guo +5

We investigate why deep neural networks suffer from loss of plasticity in continual learning, and thus fail to learn new tasks without reinitializing parameters. We show that this…