1 citations · 2 across the 13 of their papers we have counts for
7 papers · 1 filter
REVO: Rollout-Efficient Off-Policy Distillation via Variance-Guided Reuse
Yuxiao Yang, Shangzhe Li, Tianrun Yu +3
On-policy distillation (OPD) trains language models using dense token-level teacher supervision on student-generated trajectories. However, its reliance on frequently refreshed stu…
An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning
Shangzhe Li, Yuxiao Yang, Tianrun Yu +4
We study on-policy distillation (OPD) through the lens of reinforcement learning, establishing a connection between the reverse-KL objective in OPD and KL-regularized policy optimi…
Rethinking Training-Inference Mismatch in LLM Reinforcement Learning: Where It Arises and How to Correct It
Tianrun Yu, Kaixiang Zhao, Shangzhe Li +4
We study training-inference mismatch in reinforcement learning with verifiable rewards (RLVR) for large language models, where rollouts are sampled by an inference engine while gra…
When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation
Yuxiao Yang, Tianrun Yu, Shangzhe Li +6
We study length inflation in on-policy distillation (OPD), where student responses can become excessively long and even exhaust the generation budget. We identify \emph{termination…
LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation
Tianrun Yu, Kaixiang Zhao, Chih-Chun Chen +5
We study trajectory selection for reasoning distillation, where teacher-generated reasoning trajectories are selectively used as supervision for a student model. Existing methods r…
Provable and Practical In-Context Policy Optimization for Self-Improvement
Tianrun Yu, Yuxiao Yang, Zhaoyang Wang +6
We study test-time scaling, where a model improves its answer through multi-round self-reflection at inference. We introduce In-Context Policy Optimization (ICPO), in which an agen…
When the Server Steps In: Calibrated Updates for Fair Federated Learning
Tianrun Yu, Kaixiang Zhao, Cheng Zhang +4
Federated learning (FL) has emerged as a transformative distributed learning paradigm, enabling multiple clients to collaboratively train a global model under the coordination of a…