2 papers
cs.LG2026
Training Specialist Models without Reasoning Trajectories for Domain Expert Distillation
Yilei Tu, Zihao Li, Shaoxiong Ji +2
Specialist distillation effectively transfers domain expertise to student models via teacher-generated reasoning trajectories. However, when these specialists are trained solely on…
cs.AI2026
Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models
Fei Yuan, Changjiang Gao, Yilei Tu +3
Fine-tuning instruct models often improves target performance while inducing behavioral drift from the reference model, which can degrade existing capabilities. Rather than treatin…