Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Beyond LoRA: Is Sparsity-Induced Adaptation Better?
Elijah Cadenhead, Cristian McGee, Xin Li +2
Low-rank adaptation (LoRA) and its variants provide a memory- and compute-efficient alternative to full fine-tuning of pre-trained models. However, questions remain about the compa…
cs.CV2026
DnA: Denoising Attention for Visual Tasks
Ron Campos, Subhajit Maity, Xin Li +2
The softmax activation in multihead attention (MHA) is the de facto standard for attention-based models in visual perception tasks. However, standard softmax can produce noisy atte…