Sparsity May Be All You Need: Sparse Random Parameter Adaptation

Jesus Rios, Pierre Dognin, Ronny Luss +1

Full fine-tuning of large language models for alignment and task adaptation has become prohibitively expensive as models have grown in size. Parameter-Efficient Fine-Tuning (PEFT)…