Recovering General Capabilities via Uncertainty-Calibrated Multi-Teacher On-Policy Distillation

Ziyuan Liu, Jiao Ou, Jian Liang +2

Specializing large language models to vertical domains improves domain-specific behavior but often degrades general capabilities. We study this trade-off in Multi-Teacher On-Policy…