2 papers
cs.CL2024
Channel Merging: Preserving Specialization for Merged Experts
Mingyang Zhang, Jing Liu, Ganggui Ding +3
Lately, the practice of utilizing task-specific fine-tuning has been implemented to improve the performance of large language models (LLM) in subsequent tasks. Through the integrat…
cs.CL2024
ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models
Jing Liu, Ruihao Gong, Mingyang Zhang +3
LLM development involves pre-training a foundation model on massive data, followed by fine-tuning on task-specific data to create specialized experts. Serving these experts can pos…