Why Do More Experts Fail? A Theoretical Analysis of Model Merging

Zijing Wang, Xingle Xu, Yongkang Liu +6

Model merging dramatically reduces storage and computational resources by combining multiple expert models into a single multi-task model. Although recent model merging methods hav…