PACT: Preserving Anchored Cores in Task-vectors for Model Merging

Ningyuan Shi, Zhipeng Zhou, Hao Wang +2

Model merging has emerged as a training-free alternative to multi-task learning, aiming to combine multiple task-specific fine-tuned models into a single multi-task model. Most exi…