Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning

Tieliang Gong, Zhongbo Zhang, Wen Wen +1

Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies…