Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization

Xinhao Yao, Hongjin Qian, Xiaolin Hu +5

Large Language Models (LLMs), built on Transformer architectures, exhibit remarkable generalization across a wide range of tasks. However, fine-tuning these models for specific tas…