5 papers · 1 filter
Cross-Behavior Policy Optimization: Unified Learning from Offline Demonstrations and On- and Off-Policy Rollouts via a Shared Regression Target
Du Cheng, Junyu Lu, Jingfan Zhang +3
Reasoning post-training draws on offline demonstrations and policy-generated rollouts, yet conventional supervised fine-tuning (SFT) followed by reinforcement learning (RL) separat…
OPTS-TTPO: Enhancing Finite-Sample Policy-Gradient Learning with Tree Search
Junyu Lu, Shichao Weng, Zhiqiang Wang +10
The policy-gradient theorem gives the exact gradient under the current policy, but finite on-policy samples may miss rare high-return trajectories. We study whether tree search imp…
BUDDY: BUdget-Driven DYnamic Depth Routing for Adaptive Large Language Model Inference
Yuhua Zhou, Shaoqi Yu, Shichao Weng +4
Large language models (LLMs) incur high inference cost due to their depth and parameter scale. Depth pruning can reduce latency by skipping redundant Transformer blocks, but existi…
From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs
Hanmeng Liu, Shichao Weng, Xiulai Liu +3
Multiple-choice reasoning benchmarks face dual challenges: rapid saturation from advancing models and data contamination that undermines static evaluations. Ad-hoc hardening method…
CoRE: A Fine-Grained Code Reasoning Benchmark Beyond Output Prediction
Jun Gao, Yun Peng, Qian Qiao +6
Despite strong performance on code generation tasks, it remains unclear whether large language models (LLMs) genuinely reason about code execution. Existing code reasoning benchmar…