GRRR: The Geometry of Reshaping, Rotation, and Routing in Decoder LLM post-training

Jianing Qi, Hao Tang, Zhigang Zhu

We study how post-training changes the weights of Large Language Models (LLMs) relative to their pretrained weights. Across 12 post-training chains with supervised fine-tuning (SFT…