4 papers · 1 filter
Building Multi-Task Agentic LLMs via Two-Phase Distillation
Huaijie Wang, Shusheng Xu, Yi Wu +1
A key step toward artificial general intelligence is to train models that can perform multiple tasks. In this paper, we study how to build such models by first training separate RL…
AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning
Wei Fu, Jiaxuan Gao, Xujie Shen +10
Reinforcement learning (RL) has become a dominant paradigm for training large language models (LLMs), particularly for reasoning tasks. Effective RL for LLMs requires massive paral…
Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models
Xingtai Lv, Youbang Sun, Kaiyan Zhang +8
State Space Models (SSMs) have emerged as a promising alternative to the popular transformer-based models and have been increasingly gaining attention. Compared to transformers, SS…
On Designing Effective RL Reward at Training Time for LLM Reasoning
Jiaxuan Gao, Shusheng Xu, Wenjie Ye +6
Reward models have been increasingly critical for improving the reasoning capability of LLMs. Existing research has shown that a well-trained reward model can substantially improve…