Reset It and Forget It: Relearning Last-Layer Weights Improves Continual and Transfer Learning

Lapo Frati, Neil Traft, Jeff Clune +1

This work identifies a simple pre-training mechanism that leads to representations exhibiting better continual and transfer learning. This mechanism -- the repeated resetting of we…