4 papers
Parameter-Efficient Fine-Tuning with Learnable Rank
Arpit Garg, Simon Lucey, Hemanth Saratchandran
Low-Rank Adaptation (LoRA) is a popular parameter-efficient fine-tuning (PEFT) method that restricts weight updates to low-rank adapters, introducing a fixed low-rank inductive bia…
Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting
Runze Xu, Arpit Garg, Hemanth Saratchandran +1
Low-Rank Adaptation (LoRA) has become one of the most widely used fine-tuning mechanisms for adapting large language models to new domains, tasks, and users. Yet adaptation perform…
STRIDE: Training-Free Diversity Guidance via PCA-Directed Feature Perturbation in Single-Step Diffusion Models
Ankit Yadav, Arpit Garg, Ta Duc Huy +1
Distilled one-step (T=1) or few-step (T4) diffusion models enable real-time image generation but often exhibit reduced sample diversity compared to their multi-step counterpa…
Stable Forgetting: Bounded Parameter-Efficient Unlearning in Foundation Models
Arpit Garg, Hemanth Saratchandran, Ravi Garg +1
Machine unlearning in foundation models (e.g., language and vision transformers) is essential for privacy and safety; however, existing approaches are unstable and unreliable. A wi…