Stabilizing LLM Supervised Fine-Tuning via Explicit Distributional Control

Xinyu Wang, Changzhi Sun, Yuanbin Wu +1

Post-training large language models (LLMs) often suffers from catastrophic forgetting, where improvements on a target objective degrade previously acquired capabilities. Recent evi…