LoKI: Low-damage Knowledge Implanting of Large Language Models

Runyu Wang, Peng Ping, Zhengyu Guo +4

Fine-tuning adapts pretrained models for specific tasks but poses the risk of catastrophic forgetting (CF), where critical knowledge from pretraining is overwritten. To address the…