12 citations · 15 across the 6 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2024★ 2 cited
Scaling Laws for Forgetting When Fine-Tuning Large Language Models
Damjan Kalajdzievski
We study and quantify the problem of forgetting when fine-tuning pre-trained large language models (LLMs) on a downstream task. We find that parameter-efficient fine-tuning (PEFT)…
cs.CL2023★ 12 cited
A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA
Damjan Kalajdzievski
As large language models (LLMs) have become increasingly compute and memory intensive, parameter-efficient fine-tuning (PEFT) methods are now a common strategy to fine-tune LLMs. A…