Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Beyond Zero Initialization: Investigating the Impact of Non-Zero Initialization on LoRA Fine-Tuning Dynamics
Shiwei Li, Xiandi Luo, Xing Tang +6
Low-rank adaptation (LoRA) is a widely used parameter-efficient fine-tuning method. In standard LoRA layers, one of the matrices, or , is initialized to zero, ensuring that…
cs.LG2025
Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
Tai An, Ruwu Cai, Yanzhe Zhang +6
In the era of large language models (LLMs), N:M sparsity has emerged as a structured compression technique critical for accelerating inference. While prior work has primarily focus…