4 papers
Sparsity, Superposition, and Forgetting: A Mechanistic Study of Representation Retention in Continual Learning
Jan Wasilewski, JÄdrzej Kozal, MichaÅ Woźniak +1
Continual learning (CL) systems often forget previously acquired knowledge, yet the mechanisms driving forgetting remain hard to isolate in practice because real datasets entangle…
What is the role of memorization in Continual Learning?
JÄdrzej Kozal, Jan Wasilewski, Alif Ashrafee +2
Memorization impacts the performance of deep learning algorithms. Prior works have studied memorization primarily in the context of generalization and privacy. This work studies th…
Balanced Gradient Sample Retrieval for Enhanced Knowledge Retention in Proxy-based Continual Learning
Hongye Xu, Jan Wasilewski, Bartosz Krawczyk
Continual learning in deep neural networks often suffers from catastrophic forgetting, where representations for previous tasks are overwritten during subsequent training. We propo…
Continual Learning with Weight Interpolation
JÄdrzej Kozal, Jan Wasilewski, Bartosz Krawczyk +1
Continual learning poses a fundamental challenge for modern machine learning systems, requiring models to adapt to new tasks while retaining knowledge from previous ones. Addressin…