5 citations · 6 across the 10 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024★ 1 cited
Grams: Gradient Descent with Adaptive Momentum Scaling
Yang Cao, Xiaoyu Li, Zhao Song
We introduce radient Descent with daptive omentum caling (), a novel optimization algorithm that decouples the direc…
cs.LG2024
SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models
Yang Cao, Zhao Song
In this paper, we propose Singular Values and Orthonormal Regularized Singular Vectors Adaptation, or SORSA, a novel parameter efficient fine-tuning (PEFT) method. Each SORSA adapt…