Preserving Plasticity in Continual Learning with Adaptive Linearity Injection

Seyed Roozbeh Razavi Rohani, Khashayar Khajavi, Wesley Chung +2

Loss of plasticity in deep neural networks is the gradual reduction in a model's capacity to incrementally learn and has been identified as a key obstacle to learning in non-statio…