20 papers
FAPO: Flawed-Aware Policy Optimization for Efficient and Reliable Reasoning
Yuyang Ding, Chi Zhang, Juntao Li +2
Reinforcement learning with verifiable rewards (RLVR) has emerged as a promising paradigm for enhancing the reasoning capabilities of large language models (LLMs). In this context,…
SRT: Accelerating Reinforcement Learning via Speculative Rollout with Tree-Structured Cache
Chi-Chih Chang, Siqi Zhu, Zhichen Zeng +5
We present Speculative Rollout with Tree-Structured Cache (SRT), a simple, model-free approach to accelerate on-policy reinforcement learning (RL) for language models without sacri…
Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization
Xueyun Tian, Minghua Ma, Bingbing Xu +6
Supervised fine-tuning (SFT) on chain-of-thought (CoT) trajectories demonstrations is a common approach for enabling reasoning in large language models. Standard practices typicall…
MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
Chao Jin, Ziheng Jiang, Zhihao Bai +16
We present MegaScale-MoE, a production system tailored for the efficient training of large-scale mixture-of-experts (MoE) models. MoE emerges as a promising architecture to scale l…
Boosting Embodied AI Agents through Perception-Generation Disaggregation and Asynchronous Pipeline Execution
Shulai Zhang, Ao Xu, Quan Chen +6
Embodied AI systems operate in dynamic environments, requiring seamless integration of perception and generation modules to process high-frequency input and output demands. Traditi…
Verify Distributed Deep Learning Model Implementation Refinement with Iterative Relation Inference
Zhanghan Wang, Ding Ding, Hang Zhu +2
Distributed machine learning training and inference is common today because today's large models require more memory and compute than can be provided by a single GPU. Distributed m…