Heads collapse, features stay: Why Replay needs big buffers

Giulia Lanzillotta, Damiano Meier, Thomas Hofmann

A persistent paradox in continual learning (CL) is that neural networks often retain linearly separable representations of past tasks even when their output predictions fail. We fo…