Activation-Informed Merging of Large Language Models

Amin Heyrani Nobari, Kaveh Alim, Ali ArjomandBigdeli +3

Model merging, a method that combines the parameters and embeddings of multiple fine-tuned large language models (LLMs), offers a promising approach to enhance model performance ac…