Directions of Curvature as an Explanation for Loss of Plasticity

Alex Lewandowski, Haruto Tanaka, Dale Schuurmans +1

Loss of plasticity is a phenomenon in which neural networks lose their ability to learn from new experience. Despite being empirically observed in several problem settings, little…