10 papers
AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning
Bingxiang He, Pengqi Sun, Shihao Peng +18
Language model agents increasingly automate post-training, but when given the full training stack they converge on supervised fine-tuning and spend most effort revising training da…
Diffusion Reward Models
Xiangyang Wang, Bingxiang He, Zeyuan Liu +14
Reward models underpin the alignment of large language models, yet the dominant designs reduce each prompt--response pair to a point estimate or to a distribution from a fixed para…
Rethinking On-Policy Distillation of Large Language Models II: One Training Example
Zixuan Fu, Bingxiang He, Yuxin Zuo +10
On-policy distillation (OPD) combines student-generated rollouts with dense token-level supervision from a teacher. Existing work has mainly studied its algorithmic behavior, leavi…
M100: An Orchestrated Dataflow Architecture Powering General AI Computing
Yan Xie, Changkui Mao, Changsong Wu +34
As deep learning-based AI technologies gain momentum, the demand for general-purpose AI computing architectures continues to grow. While GPGPU-based architectures offer versatility…
Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
Yaxuan Li, Yuxin Zuo, Bingxiang He +8
On-policy distillation (OPD) has become a core technique in the post-training of large language models, yet its training dynamics remain poorly understood. This paper provides a sy…
How Far Can Unsupervised RLVR Scale LLM Training?
Bingxiang He, Yuxin Zuo, Zeyuan Liu +18
Unsupervised reinforcement learning with verifiable rewards (URLVR) offers a pathway to scale LLM training beyond the supervision bottleneck by deriving rewards without ground trut…
NOVA: NOise-aware Verbal Confidence CAlibration for Robust Large Language Models in RAG Systems
Jiayu Liu, Rui Wang, Qing Zong +9
Accurately assessing model confidence is essential for deploying large language models (LLMs) in mission-critical factual domains. While retrieval-augmented generation (RAG) is wid…
JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
Bingxiang He, Zekai Qu, Zeyuan Liu +9
Recent advances in reinforcement learning for large language models have converged on increasing complexity: multi-stage training pipelines, dynamic hyperparameter schedules, and c…
CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agents
Jiayu Liu, Cheng Qian, Zhaochen Su +4
Current evaluations of Large Language Model (LLM) agents primarily emphasize task completion, often overlooking resource efficiency and adaptability. This neglects a crucial capabi…
AIR: A Systematic Analysis of Annotations, Instructions, and Response Pairs in Preference Dataset
Bingxiang He, Wenbin Zhang, Jiaxi Song +11
Preference learning is critical for aligning large language models (LLMs) with human values, yet its success hinges on high-quality datasets comprising three core components: Prefe…