Do Neural Networks Lose Plasticity in a Gradually Changing World?

Tianhui Liu, Lili Mou

Continual learning has become a trending topic in machine learning. Recent studies have discovered an interesting phenomenon called loss of plasticity, referring to neural networks…