Parameter-Efficient Subspace Optimization for LLM Fine-Tuning

Yuchen Lou, Zeqi Ye, Minshuo Chen

This paper develops a new perspective on parameter-efficient fine-tuning (PEFT) for LLMs, inspired by classical subspace minimization. We introduce a unifying framework, Parameter-…