Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation

Thomas Gauthier-Caron, Shamane Siriwardhana, Elliot Stein +5

By merging models, AI systems can combine the distinct strengths of separate language models, achieving a balance between multiple capabilities without requiring substantial retrai…