9 papers
Start Classifying: Categorical Critics for LLM Reinforcement Learning
Zhijian Zhou, Long Li, Xuan Zhang +7
Proximal Policy Optimization (PPO) for large language models typically trains its critic by mean-squared-error (MSE) regression on scalar value targets. Although scalar MSE is stat…
Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning
Xuan Zhang, Zhijian Zhou, Lingfeng Qiao +6
Large language model (LLM) agents have demonstrated strong capability in sequential decision-making, yet they remains fundamentally reactive in long-horizon tasks. Unlike humans wh…
DyJR: Preserving Diversity in Reinforcement Learning with Verifiable Rewards via Dynamic Jensen-Shannon Replay
Long Li, Zhijian Zhou, Tianyi Wang +7
While Reinforcement Learning (RL) enhances Large Language Model reasoning, on-policy algorithms like GRPO are sample-inefficient as they discard past rollouts. Existing experience…
InterveneBench: Benchmarking LLMs for Intervention Reasoning and Causal Study Design in Real Social Systems
Shaojie Shi, Zhengyu Shi, Lingran Zheng +15
Causal inference in social science relies on end-to-end, intervention-centered research-design reasoning grounded in real-world policy interventions, but current benchmarks fail to…
CTkvr: KV Cache Retrieval for Long-Context LLMs via Centroid then Token Indexing
Kuan Lu, Shuhang Lin, Sai Wu +7
Large language models (LLMs) are increasingly applied in long-context scenarios such as multi-turn conversations. However, long contexts pose significant challenges for inference e…
Count Counts: Motivating Exploration in LLM Reasoning with Count-based Intrinsic Rewards
Xuan Zhang, Ruixiao Li, Zhijian Zhou +7
Reinforcement Learning (RL) has become a compelling way to strengthen the multi step reasoning ability of Large Language Models (LLMs). However, prevalent RL paradigms still lean o…