Distilling Linearized Behavior into Non-Linear Fine-Tuning for Effective Task Arithmetic

Thomas Sommariva, Francesca Morandi, Simone Calderara +1

Task vector composition has emerged as a promising paradigm for editing pre-trained models, enabling model merging through addition and unlearning through subtraction. Fine-tuning…