110 citations · 548 across the 51 of their papers we have counts for
8 papers · 1 filter
Does On-Policy Distillation Need to Complete Every Rollout?
Zhengxin Zhang, Maosen Tang, Ning Wang +6
On-policy distillation (OPD) transfers teacher knowledge to a student through feedback on student-generated rollouts, but generating complete rollouts and computing per-token teach…
Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning
Yanxi Chen, Weijie Shi, Yuexiang Xie +6
This work presents a general framework for training large language models (LLMs) to "Connect the Dots" (CoD), a meta-capability required by long-lifecycle agents: as an LLM-based A…
Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling
Yucheng Li, Huiqiang Jiang, Yang Xu +14
Reinforcement learning (RL) has become a key component in modern large language models, yet the rollout stage remains the key bottleneck in RL training pipelines. Although Multi-To…
Stabilizing Reinforcement Learning with LLMs: Formulation and Practices
Chujie Zheng, Kai Dang, Bowen Yu +10
This paper proposes a novel formulation for reinforcement learning (RL) with large language models, explaining why and under what conditions the true sequence-level reward can be o…
Soft Adaptive Policy Optimization
Chang Gao, Chujie Zheng, Xiong-Hui Chen +7
Reinforcement learning (RL) plays an increasingly important role in enhancing the reasoning capabilities of large language models (LLMs), yet stable and performant policy optimizat…
Group Sequence Policy Optimization
Chujie Zheng, Shixuan Liu, Mingze Li +9
This paper introduces Group Sequence Policy Optimization (GSPO), our stable, efficient, and performant reinforcement learning algorithm for training large language models. Unlike p…
Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Zihan Qiu, Zeyu Huang, Bo Zheng +7
This paper revisits the implementation of oad-alancing oss (LBL) when training Mixture-of-Experts (MoEs) models. Specifically, LBL for MoEs is d…
Rotated Runtime Smooth: Training-Free Activation Smoother for accurate INT4 inference
Ke Yi, Zengke Liu, Jianwei Zhang +4
Large language models have demonstrated promising capabilities upon scaling up parameters. However, serving large language models incurs substantial computation and memory movement…