4 papers
Recursive Self-Improvement via On-Policy Distillation for Reasoning
Shangjian Yin, Zehao Zhao, Kavosh Asadi +7
On-policy distillation (OPD) trains a student model by having it generate trajectories, then matching its next-token predictions with an external teacher's next-token predictions.…
Where and What to Reward? Understanding Generalization Between Verifiable and Open-Ended Domains in Reinforcement Learning
Shangjian Yin, Yu Fu, Yue Dong +1
Reinforcement learning with verifiable rewards (RLVR) is widely used to improve the reasoning capability of language models, since model-generated answers can be easily verified by…
Aligning Large Language Models via Fully Self-Synthetic Data
Shangjian Yin, Zhepei Wei, Xinyu Zhu +2
Traditional reinforcement learning from human feedback (RLHF) for large language models (LLMs) relies on expensive human-annotated datasets, while Reinforcement Learning from AI Fe…
PIKA: Expert-Level Synthetic Datasets for Post-Training Alignment from Scratch
Shangjian Yin, Shining Liang, Wenbiao Ding +4
High-quality instruction data is critical for LLM alignment, yet existing open-source datasets often lack efficiency, requiring hundreds of thousands of examples to approach propri…