Escaping Low-Dimensional Overlap: Multi-Task Model Merging via High-Dimensional Sparse Disentanglement

Yihang Zhang, Shengke Sun, Junjie Wen +1

Model merging provides an efficient way to construct multi-task generalist models without additional training, but its performance often degrades under severe task interference. Ta…