1 citations · 1 across the 10 of their papers we have counts for
10 papers
To Retain or to Adapt? Generalizing Continual Learning
Giulia Lanzillotta, Mandana Samiei, Doina Precup +2
The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting. This objective rests on an often unstated assumption: that a lifelong…
When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning
Tanguy Dieudonné, Giulia Lanzillotta, Enis Simsar +2
Low-Rank Adaptation (LoRA) has become the standard tool for parameter-efficient fine-tuning of large pretrained models. When applied sequentially across tasks in Continual Learning…
Heads collapse, features stay: Why Replay needs big buffers
Giulia Lanzillotta, Damiano Meier, Thomas Hofmann
A persistent paradox in continual learning (CL) is that neural networks often retain linearly separable representations of past tasks even when their output predictions fail. We fo…
Revisiting Knowledge Distillation: The Hidden Role of Dataset Size
Giulia Lanzillotta, Felix Sarnthein, Gil Kur +2
The concept of knowledge distillation (KD) describes the training of a student model from a teacher model and is a widely adopted technique in deep learning. However, it is still n…
Barriers for Learning in an Evolving World: Mathematical Understanding of Loss of Plasticity
Amir Joudaki, Giulia Lanzillotta, Mohammad Samragh Razlighi +5
Deep learning models excel in stationary data but struggle in non-stationary environments due to a phenomenon known as loss of plasticity (LoP), the degradation of their ability to…
Reactivation: Empirical NTK Dynamics Under Task Shifts
Yuzhi Liu, Zixuan Chen, Zirui Zhang +2
The Neural Tangent Kernel (NTK) offers a powerful tool to study the functional dynamics of neural networks. In the so-called lazy, or kernel regime, the NTK remains static during t…
The Importance of Being Lazy: Scaling Limits of Continual Learning
Jacopo Graldi, Alessandro Breccia, Giulia Lanzillotta +2
Despite recent efforts, neural networks still struggle to learn in non-stationary environments, and our understanding of catastrophic forgetting (CF) is far from complete. In this…
Local vs Global continual learning
Giulia Lanzillotta, Sidak Pal Singh, Benjamin F. Grewe +1
Continual learning is the problem of integrating new information in a model while retaining the knowledge acquired in the past. Despite the tangible improvements achieved in recent…
Towards guarantees for parameter isolation in continual learning
Giulia Lanzillotta, Sidak Pal Singh, Benjamin F. Grewe +1
Deep learning has proved to be a successful paradigm for solving many challenges in machine learning. However, deep neural networks fail when trained sequentially on multiple tasks…
Structure by Architecture: Structured Representations without Regularization
Felix Leeb, Guilia Lanzillotta, Yashas Annadani +3
We study the problem of self-supervised structured representation learning using autoencoders for downstream tasks such as generative modeling. Unlike most methods which rely on ma…