Complementing reinforcement learning with SFT through logit averaging in the post training of LLMs

Xingwei Gan, Ying Zhu

We introduce a novel method that averages the logits of a frozen reference policy (e.g., SFT) and a trainable policy, and incorporate the method into Group Relative Policy Optimiza…