4 papers · 1 filter
A Unified Generalization Framework for Model Merging: Trade-offs, Non-Linearity, and Scaling Laws
Qinglun Li, Anke Tang, Miao Zhang +3
Model merging efficiently aggregates capabilities from multiple fine-tuned models into a single one, operating purely in parameter space without original data or expensive re-compu…
Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging
Anke Tang, Enneng Yang, Li Shen +4
Deep model merging represents an emerging research direction that combines multiple fine-tuned models to harness their specialized capabilities across different tasks and domains.…
Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent
Yongxian Wei, Anke Tang, Li Shen +3
Merging multiple expert models offers a promising approach for performing multi-task learning without accessing their original data. Existing methods attempt to alleviate task conf…
Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging
Li Shen, Anke Tang, Enneng Yang +6
Multi-task learning (MTL) leverages a shared model to accomplish multiple tasks and facilitate knowledge transfer. Recent research on task arithmetic-based MTL demonstrates that me…