Predicting and Repairing Merge Collapse in Large Language Models

Jungseob Lee, Seungyoon Lee, Sugyeong Eo +3

Large language models fine-tuned from a shared base can be merged by averaging their task vectors, but some merges collapse far below the base model, and common merge operators giv…