AdaFish: Fast low-rank parameter-efficient fine-tuning by using second-order information

Jiang Hu, Quanzheng Li

Recent advancements in large-scale pretrained models have significantly improved performance across a variety of tasks in natural language processing and computer vision. However,…