Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025
Towards Higher Effective Rank in Parameter-efficient Fine-tuning using Khatri--Rao Product
Paul Albert, Frederic Z. Zhang, Hemanth Saratchandran +2
Parameter-efficient fine-tuning (PEFT) has become a standard approach for adapting large pre-trained models. Amongst PEFT methods, low-rank adaptation (LoRA) has achieved notable s…
cs.LG2025
SineLoRA: Sine-Activated Delta Compression
Cameron Gordon, Yiping Ji, Hemanth Saratchandran +2
Resource-constrained weight deployment is a task of immense practical importance. Recently, there has been interest in the specific task of \textit{Delta Compression}, where partie…
cs.LG2024
Knowledge Composition using Task Vectors with Learned Anisotropic Scaling
Frederic Z. Zhang, Paul Albert, Cristian Rodriguez-Opazo +2
Pre-trained models produce strong generic representations that can be adapted via fine-tuning. The learned weight difference relative to the pre-trained model, known as a task vect…