Continual Learning in Deep Networks: an Analysis of the Last Layer

Timothée Lesort, Thomas George, Irina Rish

We study how different output layer parameterizations of a deep neural network affects learning and forgetting in continual learning settings. The following three effects can cause…