12 papers
CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA
Gengyu Zhang, Haiyin Ran, Zhengbao He +4
As the scale of large pre-trained models continues to grow, fine-tuning them under limited memory budgets has become increasingly challenging. Low-Rank Adaptation (LoRA), currently…
SAGE: Retain-Aware Post-Hoc Sanitization of Final Unlearning Vector
Jingyuan Zhang, Yucheng Bai, Peixi Wen +6
Large Language Model (LLM) unlearning aims to remove undesirable knowledge or behaviors while preserving retained capabilities. Current unlearning methods all involve a trade-off b…
Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter
Zhengbao He, Ruiqi Ding, Zhehao Huang +3
Low-rank adaptation (LoRA) enables parameter-efficient specialization of foundation models, but the proliferation of task-specific adapters fragments capabilities across many adapt…
Bi-LoRA: Efficient Sharpness-Aware Minimization for Fine-Tuning Large-Scale Models
Yuhang Liu, Tao Li, Zhehao Huang +2
Fine-tuning large-scale pre-trained models with limited data presents significant challenges for generalization. While Sharpness-Aware Minimization (SAM) has proven effective in im…
VL-RouterBench: A Benchmark for Vision-Language Model Routing
Zhehao Huang, Baijiong Lin, Jingyuan Zhang +5
Multi-model routing has evolved from an engineering technique into essential infrastructure, yet existing work lacks a systematic, reproducible benchmark for evaluating vision-lang…
Remaining-data-free Machine Unlearning by Suppressing Sample Contribution
Xinwen Cheng, Zhehao Huang, Wenxin Zhou +4
Machine unlearning (MU) aims to remove the influence of specific training samples from a well-trained model, a task of growing importance due to the ``right to be forgotten.'' The…