2 papers
cs.CL2025
FURINA: Free from Unmergeable Router via LINear Aggregation of mixed experts
Jiayi Han, Liang Du, Yinda Chen +3
The Mixture of Experts (MoE) paradigm has been successfully integrated into Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning (PEFT), delivering performance gains with…
cs.LG2025
SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture
Jiayi Han, Liang Du, Hongwei Du +4
Although many efforts have been made, it is still a challenge to balance the training budget, downstream performance, and the general capabilities of the LLMs in many applications.…