3 papers
Zero-Shot Quantization via Weight-Space Arithmetic
Daniele Solombrino, Antonio Andrea Gargiulo, Alessandro Zirilli +3
We show that robustness to post-training quantization (PTQ) is a transferable direction in weight space. We call this direction the quantization vector: extracted from a donor task…
MASS: MoErging through Adaptive Subspace Selection
Donato Crisostomi, Alessandro Zirilli, Antonio Andrea Gargiulo +5
Model merging has recently emerged as a lightweight alternative to ensembling, combining multiple fine-tuned models into a single set of parameters with no additional training over…
Task Singular Vectors: Reducing Task Interference in Model Merging
Antonio Andrea Gargiulo, Donato Crisostomi, Maria Sofia Bucarelli +3
Task Arithmetic has emerged as a simple yet effective method to merge models without additional training. However, by treating entire networks as flat parameter vectors, it overloo…