15 papers
AlphaBet: Reinforcement Learning for LLM Event Forecasting Agents
Liqin Ye, Rongzhi Zhang, Xin Liu +5
Forecasting an unresolved event requires finding relevant evidence and deciding how strongly it supports each outcome. Yet outcome-based training of Large Language Model (LLM) fore…
Beyond the Verdict: Agentic Code Judging with Verifiable Feedback
Zheye Deng, Sha Li, Changlong Yu +4
Test-time scaling for large language models (LLMs) can involve generating multiple candidate solutions for selection or revising an earlier attempt. To guide selection and revision…
STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules
Ranajoy Sadhukhan, Euijin Hong, Sheng Cao +9
Sparse computation can scale the parametric capacity of LLMs without proportionally increasing active computation, but pretraining sparse architectures remains expensive and data-h…
Rufus-Air: An Open LLM Post-Training Recipe
Chia-Yuan Chang, Renyuan Cheng, Rui Feng +18
Rufus-Air is an open and reproducible post-training recipe on GLM-4.5-Air-Base (106B-A12B), organized as a serial pipeline of eight stages: SFT, Reasoning RL, Coding RL, Instructio…
CounterRoute: Self-Routed Reasoning via Hierarchical Counterfactual Credit Assignment
Ruochen Jiao, Besnik Fetahu, Zhenyu Shi +1
Reasoning-capable language models often produce long chains of thought when direct answers suffice, wasting inference compute. Dual-mode models offer both thinking and direct-answe…
Shopping Reasoning Bench: An Expert-Authored Benchmark for Multi-Turn Conversational Shopping Assistants
Shuxian Fan, Seonwoo Min, Youna Hu +7
Conversational shopping assistants now serve hundreds of millions of customers, yet no existing benchmark jointly evaluates the open-ended multi-turn reasoning, domain expertise, a…
Unlocking Latent Value: Taxonomy-Guided Recovery of High-Performing Data from Low-Tier Web Corpora
Neeraj Varshney, Sanket Lokegaonkar, Nasser Zalmout +3
Dominant web data curation pipelines for pretraining collapse document quality into a single composite score, systematically missing high-value content along dimensions the scorer…
Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning
Pratik Jayarao, Chaitanya Dwivedi, Himanshu Gupta +5
The performance gap across languages in LLMs is well documented, and closing it natively requires pretraining or fine-tuning on corpora that, for most languages, are quite limited.…
QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards
Rongzhi Zhang, Rui Feng, Zhihan Zhang +8
Rubric-based RL is a promising route for extending reinforcement learning beyond verifiable rewards, yet existing methods optimize rubrics while treating the query distribution as…
Training LLMs for Multi-Step Tool Orchestration with Constrained Data Synthesis and Graduated Rewards
Cheng Jiayang, Xin Liu, Zhihan Zhang +8
Multi-step tool orchestration remains challenging for LLMs, as state-of-the-art models frequently fail on full sequence execution due to parameter errors. Training for these workfl…
Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning
Xintong Li, Sha Li, Rongmei Lin +10
Large reasoning models improve with more test-time computation, but often overthink, producing unnecessarily long chains-of-thought that raise cost without improving accuracy. Prio…
HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning
Weiqi Wang, Xin Liu, Binxuan Huang +13
RLVR has become a standard recipe for training LLMs on reasoning tasks with verifiable outcomes, but when rollout generation dominates the cost, efficiency hinges on which prompts…
POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
Yizhuo Chen, Xin Liu, Ruijie Wang +7
Large language models (LLMs) are typically aligned with population-level preferences, despite substantial variation across individual users. We introduce POPI, a user-level persona…
Aligning Large Language Models with Implicit Preferences from User-Generated Content
Zhaoxuan Tan, Zheng Li, Tianyi Liu +10
Learning from preference feedback is essential for aligning large language models (LLMs) with human values and improving the quality of generated responses. However, existing prefe…
Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training
Yuchen Zhuang, Jingfeng Yang, Haoming Jiang +16
Due to the scarcity of agent-oriented pre-training data, LLM-based autonomous agents typically rely on complex prompting or extensive fine-tuning, which often fails to introduce ne…