2 papers
cs.CL2025
Dual Decomposition of Weights and Singular Value Low Rank Adaptation
Jialong Han, Si Zhang, Ke Zhang
Parameter-Efficient Fine-Tuning (PEFT) has emerged as a critical paradigm for adapting Large Language Models (LLMs) to downstream tasks, among which Low-rank Adaptation (LoRA) repr…
cs.CL2025
OSoRA: Output-Dimension and Singular-Value Initialized Low-Rank Adaptation
Jialong Han, Si Zhang, Ke Zhang
Fine-tuning Large Language Models (LLMs) has become increasingly challenging due to their massive scale and associated computational costs. Parameter-Efficient Fine-Tuning (PEFT) m…