Showing 2026Show all
3 papers · 1 filter
cs.CV2026
DepthBench: Measuring How Residual Connections Enable More Computational Depth
Keyu Wang, Yangyi Huang, Jiale Kang +3
Depth is a natural way to increase the computational capacity in Transformers, yet the contribution of deeper layers can diminish as depth grows larger. Recent approaches enhance n…
cs.LG2026
Normalized Low-Rank Adaptation
Jiale Kang, Ziyin Yue, Zheng Zhan +2
While low-rank adaptation (LoRA) is widely used for parameter-efficient model adaptation, how to regularize its training dynamics for stable and effective optimization remains unde…
cs.LG2026
PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective
Yangyi Huang, Ruotian Peng, Zeju Qiu +4
Parameter-efficient finetuning (PEFT) has become the standard approach for adapting large language models, yet evaluations largely emphasize downstream accuracy while overlooking t…