11 papers
OmniReasoner: Thinking with Long Audio-Video via Native Tool Use
Yu Chen, Caorui Li, Ziyu Xiong +8
Long audio-video reasoning is difficult for omnimodal LLMs because the decisive evidence is often sparse, cross-modal, and too expensive to preserve with uniformly high-fidelity in…
Decomposing and Steering Functional Metacognition in Large Language Models
Yanshi Li, Xueru Bai, Shuman Liu +2
Large language models (LLMs) increasingly exhibit behaviors suggesting awareness of their evaluation context, often adapting their reasoning strategies in benchmark settings. Prior…
Towards On-Policy SFT: Distribution Discriminant Theory and its Applications in LLM Training
Miaosen Zhang, Yishan Liu, Shuxia Lin +8
Supervised fine-tuning (SFT) is computationally efficient but often yields inferior generalization compared to reinforcement learning (RL). This gap is primarily driven by RL's use…
ESPO: Entropy Importance Sampling Policy Optimization
Yuepeng Sheng, Yuwei Huang, Shuman Liu +2
Reinforcement learning (RL) has become a central component of post-training for large language models (LLMs), particularly for complex reasoning tasks that require stable optimizat…
SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts
Bingshuai Liu, Ante Wang, Zijun Min +7
Large Language Models (LLMs) increasingly rely on reinforcement learning with verifiable rewards (RLVR) to elicit reliable chain-of-thought reasoning. However, the training process…
Orchestrating Tokens and Sequences: Dynamic Hybrid Policy Optimization for RLVR
Zijun Min, Bingshuai Liu, Ante Wang +4
Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising framework for optimizing large language models in reasoning tasks. However, existing RLVR algorithms focus…