SuperMerge: An Approach For Gradient-Based Model Merging

Haoyu Yang, Zheng Zhang, Saket Sathe

Large language models, such as ChatGPT, Claude, or LLaMA, are gigantic, monolithic, and possess the superpower to simultaneously support thousands of tasks. However, high-throughpu…