557 citations · 832 across the 25 of their papers we have counts for
21 papers · 1 filter
Forgetting, plasticity, and co-observation: a third facet of continual learning
Timm Hess, Abhishek Jha, Gido M. van de Ven +1
Efficient continual learning remains a fundamental challenge for deep neural networks. While catastrophic forgetting and loss of plasticity are widely considered the primary obstac…
Position: Modular Memory is the Key to Continual Learning Agents
Vaggelis Dorovatas, Malte Schwerin, Andrew D. Bagdanov +21
Foundation models have transformed machine learning through large-scale pretraining and increased test-time compute. Despite surpassing human performance in several domains, these…
Revisiting Weight Regularization for Low-Rank Continual Learning
Yaoyue Zheng, Yin Zhang, Joost van de Weijer +4
Continual Learning (CL) with large-scale pre-trained models (PTMs) has recently gained wide attention, shifting the focus from training from scratch to continually adapting PTMs. T…
Putting a Face to Forgetting: Continual Learning meets Mechanistic Interpretability
Sergi Masip, Gido M. van de Ven, Javier Ferrando +1
Catastrophic forgetting in continual learning is often measured at the performance or last-layer representation level, overlooking the underlying mechanisms. We introduce a mechani…
Continual Learning Should Move Beyond Incremental Classification
Rupert Mitchell, Antonio Alliegro, Raffaello Camoriano +17
Continual learning (CL) is the sub-field of machine learning concerned with accumulating knowledge in dynamic environments. So far, CL research has mainly focused on incremental cl…
On the Computation of the Fisher Information in Continual Learning
Gido M. van de Ven
One of the most popular methods for continual learning with deep neural networks is Elastic Weight Consolidation (EWC), which involves computing the Fisher Information. The exact w…
Continual Learning in the Presence of Repetition
Hamed Hemati, Lorenzo Pellegrini, Xiaotian Duan +11
Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world p…
Continual Learning and Catastrophic Forgetting
Gido M. van de Ven, Nicholas Soures, Dhireesha Kudithipudi
This book chapter delves into the dynamics of continual learning, which is the process of incrementally learning from a non-stationary stream of data. Although continual learning i…
Infinite dSprites for Disentangled Continual Learning: Separating Memory Edits from Generalization
Sebastian Dziadzio, Çağatay Yıldız, Gido M. van de Ven +3
The ability of machine learning systems to learn continually is hindered by catastrophic forgetting, the tendency of neural networks to overwrite previously acquired knowledge when…
Continual Learning of Diffusion Models with Generative Distillation
Sergi Masip, Pau Rodriguez, Tinne Tuytelaars +1
Diffusion models are powerful generative models that achieve state-of-the-art performance in image synthesis. However, training them demands substantial amounts of data and computa…
Continual Learning: Applications and the Road Forward
Eli Verwimp, Rahaf Aljundi, Shai Ben-David +17
Continual learning is a subfield of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting wh…
Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How
Timm Hess, Tinne Tuytelaars, Gido M. van de Ven
Recent years have seen considerable progress in the continual training of deep neural networks, predominantly thanks to approaches that add replay or regularization terms to the lo…
Prediction Error-based Classification for Class-Incremental Learning
Michał Zając, Tinne Tuytelaars, Gido M. van de Ven
Class-incremental learning (CIL) is a particularly challenging variant of continual learning, where the goal is to learn to discriminate between all classes presented in an increme…
Knowledge Accumulation in Continually Learned Representations and the Issue of Feature Forgetting
Timm Hess, Eli Verwimp, Gido M. van de Ven +1
Continual learning research has shown that neural networks suffer from catastrophic forgetting "at the output level", but it is debated whether this is also the case at the level o…
Continual evaluation for lifelong learning: Identifying the stability gap
Matthias De Lange, Gido van de Ven, Tinne Tuytelaars
Time-dependent data-generating distributions have proven to be difficult for gradient-based training of neural networks, as the greedy updates result in catastrophic forgetting of…
Natural continual learning: success is a journey, not (just) a destination
Ta-Chu Kao, Kristopher T. Jensen, Gido M. van de Ven +2
Biological agents are known to learn many different tasks over the course of their lives, and to be able to revisit previous tasks and behaviors with little to no loss in performan…
Class-Incremental Learning with Generative Classifiers
Gido M. van de Ven, Zhe Li, Andreas S. Tolias
Incrementally training deep neural networks to recognize new classes is a challenging problem. Most existing class-incremental learning methods store data or use generative replay,…
Avalanche: an End-to-End Library for Continual Learning
Vincenzo Lomonaco, Lorenzo Pellegrini, Andrea Cossu +25
Learning continually from non-stationary data streams is a long-standing goal and a challenging problem in machine learning. Recently, we have witnessed a renewed and fast-growing…
Energy-Based Models for Continual Learning
Shuang Li, Yilun Du, Gido M. van de Ven +1
We motivate Energy-Based Models (EBMs) as a promising model class for continual learning problems. Instead of tackling continual learning via the use of external memory, growing mo…
Three scenarios for continual learning
Gido M. van de Ven, Andreas S. Tolias
Standard artificial neural networks suffer from the well-known issue of catastrophic forgetting, making continual or lifelong learning difficult for machine learning. In recent yea…
Generative replay with feedback connections as a general strategy for continual learning
Gido M. van de Ven, Andreas S. Tolias
A major obstacle to developing artificial intelligence applications capable of true lifelong learning is that artificial neural networks quickly or catastrophically forget previous…