Channel Merging: Preserving Specialization for Merged Experts

Mingyang Zhang, Jing Liu, Ganggui Ding +3

Lately, the practice of utilizing task-specific fine-tuning has been implemented to improve the performance of large language models (LLM) in subsequent tasks. Through the integrat…