Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Reinforcing What the Student Has Learned: Teacher-Guided On-Policy Distillation
Bozhen Fang, Linhao Zhong, Xuehao Wang +3
On-policy distillation (OPD) improves language-model reasoning through dense teacher feedback on student-generated responses. We first observe an amplitude disparity in this feedba…
cs.LG2024
Improve Cross-Architecture Generalization on Dataset Distillation
Binglin Zhou, Linhao Zhong, Wentao Chen
Dataset distillation, a pragmatic approach in machine learning, aims to create a smaller synthetic dataset from a larger existing dataset. However, existing distillation methods pr…