Extrapolation Merging: Keep Improving With Extrapolation and Merging

Yiguan Lin, Bin Xu, Yinghao Li +1

Large Language Models (LLMs) require instruction fine-tuning to perform different downstream tasks. However, the instruction fine-tuning phase still demands significant computation…