Activation-Efficient Fine-Tuning via Sequence-Compressed Low-Rank Subspaces

Irene Tenison, Stella Ahn, Miriam Kim +2

Parameter-Efficient Fine-Tuning (PEFT) has become the standard approach for adapting large language models (LLMs). However, despite reducing trainable parameter counts, many PEFT m…