A Unified Noise-Curvature View of Loss of Trainability

Gunbir Singh Baveja, Alex Lewandowski, Mark Schmidt

Loss of trainability refers to a phenomenon in continual learning where parameter updates no longer make progress on the optimization objective, so accuracy stalls or degrades as t…