1 citations · 1 across the 4 of their papers we have counts for
3 papers · 1 filter
Towards Higher Effective Rank in Parameter-efficient Fine-tuning using Khatri--Rao Product
Paul Albert, Frederic Z. Zhang, Hemanth Saratchandran +2
Parameter-efficient fine-tuning (PEFT) has become a standard approach for adapting large pre-trained models. Amongst PEFT methods, low-rank adaptation (LoRA) has achieved notable s…
SineLoRA: Sine-Activated Delta Compression
Cameron Gordon, Yiping Ji, Hemanth Saratchandran +2
Resource-constrained weight deployment is a task of immense practical importance. Recently, there has been interest in the specific task of \textit{Delta Compression}, where partie…
RandLoRA: Full-rank parameter-efficient fine-tuning of large models
Paul Albert, Frederic Z. Zhang, Hemanth Saratchandran +3
Low-Rank Adaptation (LoRA) and its variants have shown impressive results in reducing the number of trainable parameters and memory requirements of large transformer networks while…