Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Sparse High Rank Adapters
Kartikeya Bhardwaj, Nilesh Prasad Pandey, Sweta Priyadarshi +9
Low Rank Adaptation (LoRA) has gained massive attention in the recent generative AI research. One of the main advantages of LoRA is its ability to be fused with pretrained models,…
cs.LG2024
Rapid Switching and Multi-Adapter Fusion via Sparse High Rank Adapters
Kartikeya Bhardwaj, Nilesh Prasad Pandey, Sweta Priyadarshi +9
In this paper, we propose Sparse High Rank Adapters (SHiRA) that directly finetune 1-2% of the base model weights while leaving others unchanged, thus, resulting in a highly sparse…