CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation

Pengcheng Wang, Ziran Liu, Wei Wang +1

Parameter-Efficient Fine-Tuning (PEFT) commonly adapts pretrained weights through low-rank updates, and recent methods further exploit the singular value decomposition (SVD) of the…