Singular Value Decomposition on Kronecker Adaptation for Large Language Model

Yee Hin Chong, Peng Qu

Large pre-trained Transformer models achieve state-of-the-art results across diverse language and reasoning tasks, but full fine-tuning incurs substantial storage, memory, and comp…