What Matters for Model Merging at Scale?

Prateek Yadav, Tu Vu, Jonathan Lai +4

Model merging aims to combine multiple expert models into a more capable single model, offering benefits such as reduced storage and serving costs, improved generalization, and sup…