Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
On Catastrophic Forgetting in Low-Rank Decomposition-Based Parameter-Efficient Fine-Tuning
Muhammad Ahmad, Jingjing Zheng, Yankai Cao
Parameter-efficient fine-tuning (PEFT) based on low-rank decomposition, such as LoRA, has become a standard for adapting large pretrained models. However, its behavior in sequentia…
cs.LG2024
Adaptive Principal Components Allocation with the -regularized Gaussian Graphical Model for Efficient Fine-Tuning Large Models
Jingjing Zheng, Yankai Cao
In this work, we propose a novel Parameter-Efficient Fine-Tuning (PEFT) approach based on Gaussian Graphical Models (GGMs), marking the first application of GGMs to PEFT tasks, to…