8 citations · 11 across the 16 of their papers we have counts for
6 papers · 1 filter
Building Multi-Task Agentic LLMs via Two-Phase Distillation
Huaijie Wang, Shusheng Xu, Yi Wu +1
A key step toward artificial general intelligence is to train models that can perform multiple tasks. In this paper, we study how to build such models by first training separate RL…
AREAL-DTA: Dynamic Tree Attention for Efficient Reinforcement Learning of Large Language Models
Jiarui Zhang, Yuchen Yang, Ran Yan +8
Reinforcement learning (RL)-based post-training for large language models (LLMs) is computationally expensive, as it generates many rollout sequences that frequently share long tok…
AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning
Wei Fu, Jiaxuan Gao, Xujie Shen +10
Reinforcement learning (RL) has become a dominant paradigm for training large language models (LLMs), particularly for reasoning tasks. Effective RL for LLMs requires massive paral…
Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models
Xingtai Lv, Youbang Sun, Kaiyan Zhang +8
State Space Models (SSMs) have emerged as a promising alternative to the popular transformer-based models and have been increasingly gaining attention. Compared to transformers, SS…
On Designing Effective RL Reward at Training Time for LLM Reasoning
Jiaxuan Gao, Shusheng Xu, Wenjie Ye +6
Reward models have been increasingly critical for improving the reasoning capability of LLMs. Existing research has shown that a well-trained reward model can substantially improve…
Iteratively Learn Diverse Strategies with State Distance Information
Wei Fu, Weihua Du, Jingwei Li +3
In complex reinforcement learning (RL) problems, policies with similar rewards may have substantially different behaviors. It remains a fundamental challenge to optimize rewards wh…