Mapping Post-Training Forgetting in Language Models at Scale

Jackson Harmon, Andreas Hochlehnert, Matthias Bethge +1

Scaled post-training now drives many of the largest capability gains in language models (LMs), yet its effect on pretrained knowledge remains poorly understood. Not all forgetting…