Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling
Yupeng Chang, Yuan Wu, Yi Chang
Low-Rank Adaptation (LoRA) is a widely used parameter-efficient fine-tuning (PEFT) method for large language models. Under a fixed rank budget, LoRA parameterizes each adapted weig…
cs.LG2026
AGGC: Adaptive Group Gradient Clipping for Stabilizing Large Language Model Training
Zhiyuan Li, Yuan Wu, Yi Chang
To stabilize the training of Large Language Models (LLMs), gradient clipping is a nearly ubiquitous heuristic used to alleviate exploding gradients. However, traditional global nor…
cs.LG2025
R-LoRA: Randomized Multi-Head LoRA for Efficient Multi-Task Learning
Jinda Liu, Yi Chang, Yuan Wu
Fine-tuning large language models (LLMs) is computationally expensive, and Low-Rank Adaptation (LoRA) provides a cost-effective solution by approximating weight updates through low…