127 citations · 319 across the 41 of their papers we have counts for
16 papers · 1 filter
IrekoGPT: Turning Structured Pruning into Post-Hoc Slimmable LLMs
Pietro Moriello, Pietro Buzzega, Angelo Porrello +1
We introduce IrekoGPT, a post-hoc method for converting pretrained LLMs into slimmable models whose width can be adjusted at inference time. Building on SliceGPT, we retain its pro…
Distilling Linearized Behavior into Non-Linear Fine-Tuning for Effective Task Arithmetic
Thomas Sommariva, Francesca Morandi, Simone Calderara +1
Task vector composition has emerged as a promising paradigm for editing pre-trained models, enabling model merging through addition and unlearning through subtraction. Fine-tuning…
Transporting Task Vectors across Different Architectures without Training
Filippo Rinaldi, Aniello Panariello, Giacomo Salici +2
Adapting large pre-trained models to downstream tasks often produces task-specific parameter updates that are expensive to relearn for every model variant. While recent work has sh…
Gradient-Sign Masking for Task Vector Transport Across Pre-Trained Models
Filippo Rinaldi, Aniello Panariello, Giacomo Salici +4
When a new release of a foundation model is published, practitioners typically need to repeat fine-tuning, even if the same task was already tackled in the previous version. A prom…
Rethinking Layer-wise Model Merging through Chain of Merges
Pietro Buzzega, Riccardo Salami, Angelo Porrello +1
Fine-tuning pretrained models has become a standard pathway to achieve state-of-the-art performance across a wide range of domains, leading to a proliferation of task-specific mode…
Update Your Transformer to the Latest Release: Re-Basin of Task Vectors
Filippo Rinaldi, Giacomo Capitani, Lorenzo Bonicelli +6
Foundation models serve as the backbone for numerous specialized models developed through fine-tuning. However, when the underlying pretrained model is updated or retrained (e.g.,…
May the Forgetting Be with You: Alternate Replay for Learning with Noisy Labels
Monica Millunzi, Lorenzo Bonicelli, Angelo Porrello +3
Forgetting presents a significant challenge during incremental training, making it particularly demanding for contemporary AI systems to assimilate new knowledge in streaming data…
Semantic Residual Prompts for Continual Learning
Martin Menabue, Emanuele Frascaroli, Matteo Boschini +4
Prompt-tuning methods for Continual Learning (CL) freeze a large pre-trained model and train a few parameter vectors termed prompts. Most of these methods organize these vectors in…
Self-Labeling the Job Shop Scheduling Problem
Andrea Corsini, Angelo Porrello, Simone Calderara +1
This work proposes a self-supervised training strategy designed for combinatorial problems. An obstacle in applying supervised paradigms to such problems is the need for costly tar…
On the Effectiveness of Equivariant Regularization for Robust Online Continual Learning
Lorenzo Bonicelli, Matteo Boschini, Emanuele Frascaroli +6
Humans can learn incrementally, whereas neural networks forget previously acquired information catastrophically. Continual Learning (CL) approaches seek to bridge this gap by facil…
Input Perturbation Reduces Exposure Bias in Diffusion Models
Mang Ning, Enver Sangineto, Angelo Porrello +2
Denoising Diffusion Probabilistic Models have shown an impressive generation quality, although their long sampling chain leads to high computational costs. In this paper, we observ…
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning
Lorenzo Bonicelli, Matteo Boschini, Angelo Porrello +2
Rehearsal approaches enjoy immense popularity with Continual Learning (CL) practitioners. These methods collect samples from previously encountered data distributions in a small me…
Transfer without Forgetting
Matteo Boschini, Lorenzo Bonicelli, Angelo Porrello +5
This work investigates the entanglement between Continual Learning (CL) and Transfer Learning (TL). In particular, we shed light on the widespread application of network pretrainin…
Class-Incremental Continual Learning into the eXtended DER-verse
Matteo Boschini, Lorenzo Bonicelli, Pietro Buzzega +2
The staple of human intelligence is the capability of acquiring knowledge in a continuous fashion. In stark contrast, Deep Networks forget catastrophically and, for this reason, th…
Rethinking Experience Replay: a Bag of Tricks for Continual Learning
Pietro Buzzega, Matteo Boschini, Angelo Porrello +1
In Continual Learning, a Neural Network is trained on a stream of data whose distribution shifts over time. Under these assumptions, it is especially challenging to improve on clas…
Classifying Signals on Irregular Domains via Convolutional Cluster Pooling
Angelo Porrello, Davide Abati, Simone Calderara +1
We present a novel and hierarchical approach for supervised classification of signals spanning over a fixed graph, reflecting shared properties of the dataset. To this end, we intr…