3 papers
cs.LG2025
LoSiA: Efficient High-Rank Fine-Tuning via Subnet Localization and Optimization
Xujia Wang, Yunjia Qi, Bin Xu
Parameter-Efficient Fine-Tuning (PEFT) methods, such as LoRA, significantly reduce the number of trainable parameters by introducing low-rank decomposition matrices. However, exist…
cs.LG2025
Recurrent Knowledge Identification and Fusion for Language Model Continual Learning
Yujie Feng, Xujia Wang, Zexin Lu +7
Continual learning (CL) is crucial for deploying large language models (LLMs) in dynamic real-world environments without costly retraining. While recent model ensemble and model me…
cs.CL2024
MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning
Xujia Wang, Haiyan Zhao, Shuo Wang +2
Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA have significantly improved the adaptation of LLMs to downstream tasks in a resource-efficient manner. However, in multi-ta…