Refine Large Language Model Fine-tuning via Instruction Vector

Gangwei Jiang, Zhaoyi Li, Defu Lian +1

Fine-tuning large language models (LLMs) can cause them to lose their general capabilities. However, the intrinsic mechanisms behind such forgetting remain unexplored. In this pape…