HUT: A More Computation Efficient Fine-Tuning Method With Hadamard Updated Transformation

Geyuan Zhang, Xiaofei Zhou, Chuheng Chen

Fine-tuning pre-trained language models for downstream tasks has achieved impressive results in NLP. However, fine-tuning all parameters becomes impractical due to the rapidly incr…