1 citations · 1 across the 2 of their papers we have counts for
2 papers
cs.CL2024
DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models
Xiaolin Hu, Xiang Cheng, Peiyu Liu +4
Low-rank adaptation (LoRA) reduces the computational and memory demands of fine-tuning large language models (LLMs) by approximating updates with low-rank matrices. However, low-ra…
cs.CL2024★ 1 cited
PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning
Qibin Wang, Xiaolin Hu, Weikai Xu +3
Low-rank adaptation (LoRA) and its variants have recently gained much interest due to their ability to avoid excessive inference costs. However, LoRA still encounters the following…