Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation

Xin Li, Hao Jiang, Xin Gao +6

Reinforcement learning can turn one language model into several specialists, each excellent at a single skill such as mathematics, coding or following instructions, but users need…