TeachBack: Learning to Teach via Bidirectional Preference Optimization for On-Policy Self-Distillation

Linfei Ji, Jiayun Li, Haozhao Wang +5

On-policy self-distillation (OPSD) enables a language model to learn from a Teacher instantiated from the same model but given access to privileged information, such as a reference…