paper

Failure Modes of Always-On Inter-Cluster Repulsion in Replay-Based Continual Learning

arXiv:2510.07648

Abstract

Feature-space objectives are often added to replay-based continual learning systems with the expectation that better geometric separation will improve retention. We study a preliminary form of Cluster-Aware Replay (CAR) that combines a class-balanced replay memory with an always-active inter-cluster repulsion term (ICF). On five-task Split CIFAR-10 with a ResNet-18 backbone, the highest observed mean in a six-value sensitivity sweep reaches final average accuracy over three seeds, compared with for replay alone. ICF without replay reaches only . All tested repulsion weights produce final accuracies between and , and the detailed configuration exhibits percentage points of average forgetting. An instrumented rerun shows that the weighted repulsion contribution remains near after cross-entropy has fallen close to zero, so the total objective becomes negative while old-task accuracy collapses. Importantly, the normalized distance objective is mathematically bounded; the failure is therefore better described as non-saturating, always-on repulsion rather than an unbounded loss. These negative results show that geometric separation is not automatically complementary to replay and motivate margin-gated objectives whose gradients deactivate once sufficient separation has been reached.

arXiv commentv3: Substantially revised as a negative-results study. All reported values recomputed from verified experimental runs; method description corrected to match the released implementation; statistical aggregation corrected to per-seed population standard deviations. 4 pages, 2 figures, 3 tables. Code: https://github.com/AminHasibul/Continual-Learning-UsingInterClusterDistance