AdaGradSelect: An adaptive gradient-guided layer selection method for efficient fine-tuning of SLMs

Anshul Kumar, Gagan Raj Gupta, Manisha Chawla

Large Language Models (LLMs) can perform many NLP tasks well, but fully fine-tuning them is expensive and requires a lot of memory. Parameter-Efficient Fine-Tuning (PEFT) methods s…