11 papers
Tree-based Credit Assignment for Multi-Agent Memory System
Marina Mao, Alexandr Liu, Pengbo Li +3
Memory systems are widely adopted to enhance LLMs for long-horizon tasks, and are commonly organized as multi-agent pipelines with memory building, summarizing, and retrieval agent…
Discovery and Reinforcement of Tool-Integrated Reasoning Chains via Rollout Trees
Kun Li, Zenan Xu, Junan Li +4
Tool-Integrated Reasoning has emerged as a key paradigm to augment Large Language Models (LLMs) with computational capabilities, yet integrating tool-use into long Chain-of-Thought…
Segmental Advantage Estimation: Enhancing PPO for Long-Context LLM Training
Xue Gong, Qi Yi, Ziyuan Nan +8
Training Large Language Models (LLMs) for reasoning tasks is increasingly driven by Reinforcement Learning with Verifiable Rewards (RLVR), where Proximal Policy Optimization (PPO)…
ConMax: Confidence-Maximizing Compression for Efficient Chain-of-Thought Reasoning
Minda Hu, Zexuan Qiu, Zenan Xu +3
Recent breakthroughs in Large Reasoning Models (LRMs) have demonstrated that extensive Chain-of-Thought (CoT) generation is critical for enabling intricate cognitive behaviors, suc…
ATPO: Agentic Turn-based Policy Optimization via Tree Search
Zefang Zong, Dingwei Chen, Yang Li +6
LLM agents have emerged as powerful systems for tackling multi-turn tasks by interleaving internal reasoning and external tool interactions. Agentic Reinforcement Learning has rece…
Accelerating Scientific Discovery with Autonomous Goal-evolving Agents
Yuanqi Du, Botao Yu, Tianyu Liu +25
There has been unprecedented interest in developing agents that expand the boundary of scientific discovery, primarily by optimizing quantitative objective functions specified by s…
Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward
Guanhua Huang, Tingqiang Xu, Mingze Wang +7
Reinforcement Learning with Verifiable Rewards (RLVR) has propelled Large Language Models in complex reasoning, yet its scalability is often hindered by a training bottleneck where…
ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding
Yuhang Li, Chenchen Zhang, Ruilin Lv +6
While Large Language Models (LLMs) excel at algorithmic code generation, they struggle with front-end development, where correctness is judged on rendered pixels and interaction. W…
Reinforcement Learning on Pre-Training Data
Siheng Li, Kejiao Li, Zenan Xu +33
The growing disparity between the exponential scaling of computational resources and the finite growth of high-quality text data now constrains conventional scaling approaches for…
Adaptive Termination for Multi-round Parallel Reasoning: An Universal Semantic Entropy-Guided Framework
Zenan Xu, Zexuan Qiu, Guanhua Huang +9
Recent advances in large language models (LLMs) have accelerated progress toward artificial general intelligence, with inference-time scaling emerging as a key technique. Contempor…
Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thought
Tencent Hunyuan Team, Ao Liu, Botong Zhou +248
As Large Language Models (LLMs) rapidly advance, we introduce Hunyuan-TurboS, a novel large hybrid Transformer-Mamba Mixture of Experts (MoE) model. It synergistically combines Mam…