565 citations · 663 across the 57 of their papers we have counts for
12 papers · 1 filter
Task Singular Vectors: Reducing Task Interference in Model Merging
Antonio Andrea Gargiulo, Donato Crisostomi, Maria Sofia Bucarelli +3
Task Arithmetic has emerged as a simple yet effective method to merge models without additional training. However, by treating entire networks as flat parameter vectors, it overloo…
ATM: Improving Model Merging by Alternating Tuning and Merging
Luca Zhou, Daniele Solombrino, Donato Crisostomi +3
Model merging has emerged as a cost-efficient approximation to multitask learning. Among merging strategies, task arithmetic is notable for its simplicity and effectiveness. In thi…
ResiDual Transformer Alignment with Spectral Decomposition
Lorenzo Basile, Valentino Maiorca, Luca Bortolussi +2
When examined through the lens of their residual streams, a puzzling property emerges in transformer networks: residual contributions (e.g., attention heads) sometimes specialize i…
TOAST: Transformer Optimization using Adaptive and Simple Transformations
Irene Cannistraci, Simone Antonelli, Emanuele Palumbo +4
Foundation models achieve state-of-the-art performance across different tasks, but their size and computational demands raise concerns about accessibility and sustainability. Exist…
Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutions
Michele Miranda, Elena Sofia Ruzzetti, Andrea Santilli +3
Large Language Models (LLMs) represent a significant advancement in artificial intelligence, finding applications across various domains. However, their reliance on massive interne…
Latent Space Translation via Inverse Relative Projection
Valentino Maiorca, Luca Moschella, Marco Fumero +2
The emergence of similar representations between independently trained neural models has sparked significant interest in the representation learning community, leading to the devel…