LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Merging

Zehua Liu, Han Wu, Yuxuan Yao +4

While most current approaches rely on further training techniques, such as fine-tuning or reinforcement learning, to enhance model capacities, model merging stands out for its abil…