Block Circulant Adapter for Large Language Models

Xinyu Ding, Meiqi Wang, Siyu Liao +1

Fine-tuning large language models (LLMs) is difficult due to their huge model size. Recent Fourier domain-based methods show potential for reducing fine-tuning costs. We propose a…