Jiarui Yao

Enabling Token-wise Importance Sampling in Off-Policy RLVR with Proximal KL Regularization

Hao Bai, Jiarui Yao, Tong Zhang

Off-policy data in reinforcement learning with verifiable rewards (RLVR) can arise from asynchronous generation, rollout-training mismatch, and response reuse. Token-wise importanc…

Game-Creation-Agents: Autonomous Game Production

Tencent Multimodal RL Team, Xiangxin Zhou, Jiaqi Tang +15

Game-Creation-Agents (GCA) turns a natural-language request into a runnable Phaser 4 browser game. The pipeline first builds a multimodal game design document that couples mechanic…

LMBuild: Evaluating LLM Agents for Generating Buildable and Functional Structures

Jiateng Liu, Rushi Wang, Cheng Qian +9

LLM-based agents are increasingly capable of generating complex 3D structures, with the potential to reshape how objects are designed and realized in the physical world. Yet, produ…

cs.RO2026

MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation

Bingxuan Li, Siqi Song, Yizhuo Wu +3

Vision-language-action (VLA) models have advanced robotic manipulation, but their zero-shot generalization in new tasks and environments remains limited, and their reliance on spec…

SceneActBench: Can Agents Act on the 3D Scenes They See?

Yifei Zhao, Xiangxin Zhou, Wenhao Yang +11

Vision-language model (VLM) agents increasingly use tools to act on 3D scenes rather than only describe them. Existing 3D benchmarks score textual responses or single-object operat…

Predictive Divergence Masks for LLM RL

Xiangxin Zhou, Jiarui Yao, Penghui Qi +4

Reinforcement learning for large language models (LLMs) typically relies on trust-region masks to stabilize off-policy updates. The dominant PPO-style approach uses the sampled-tok…

GRAIN: Group Aggregation via Min-Norm Objective

Nghia Bui, Jiarui Yao, Lijing Wang

Learning instability is a long-standing problem across machine learning, but it is especially acute in the overparameterized regime that defines modern deep learning: large models…

Rethinking the Divergence Regularization in LLM RL

Jiarui Yao, Xiangxin Zhou, Penghui Qi +3

Reinforcement learning (RL) has become a key component of post-training large language models (LLMs). In practice, LLM RL is often off-policy because of training-inference mismatch…

cs.CL2026

AgentSPEX: An Agent SPecification and EXecution Language

Pengcheng Wang, Jerry Huang, Jiarui Yao +7

Language-model agent systems commonly rely on reactive prompting, in which a single instruction guides the model through an open-ended sequence of reasoning and tool-use steps, lea…

Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Models

Haitao Jiang, Wenbo Zhang, Jiarui Yao +3

Pre-trained Large Language Model (LLM) exhibits broad capabilities, yet, for specific tasks or domains their attainment of higher accuracy and more reliable reasoning generally dep…

Future-KL Regularized GRPO: Process-Level Credit Assignment from $f$ -Divergence Regularization

Jiarui Yao, Ruida Wang, Hao Bai +1

Group Relative Policy Optimization (GRPO) is widely used for critic-free Large Language Model (LLM) post-training, but its KL regularization is usually implemented as a local loss-…

cs.AI2025

ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning

Hanyang Chen, Mark Zhao, Rui Yang +15

Recent advances in embodied AI highlight the potential of vision language models (VLMs) as agents capable of perception, reasoning, and interaction in complex environments. However…

GAR: Generative Adversarial Reinforcement Learning for Formal Theorem Proving

Ruida Wang, Jiarui Yao, Rui Pan +2

Solving math problems through verifiable languages such as Lean has significantly impacted both the mathematics and computer science communities. Current state-of-the-art models ar…

cs.CL2025

Why is Your Language Model a Poor Implicit Reward Model?

Noam Razin, Yong Lin, Jiarui Yao +1

Reward models are key to language model post-training and inference pipelines. Conveniently, recent work showed that every language model defines an implicit reward model (IM-RM),…

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models

Zihan Wang, Rui Pan, Jiarui Yao +7

We propose Chain-of-Experts (CoE), a new Mixture-of-Experts (MoE) architecture that introduces sequential expert communication within each layer. Unlike traditional MoE models, whe…

Optimizing Chain-of-Thought Reasoners via Gradient Variance Minimization in Rejection Sampling and RL

Jiarui Yao, Yifan Hao, Hanning Zhang +4

Chain-of-thought (CoT) reasoning in large language models (LLMs) can be formalized as a latent variable problem, where the model needs to generate intermediate reasoning steps. Whi…

cs.AI2025

MiCRo: Mixture Modeling and Context-aware Routing for Personalized Preference Learning

Jingyan Shen, Jiarui Yao, Rui Yang +5

Reward modeling is a key step in building safe foundation models when applying reinforcement learning from human feedback (RLHF) to align Large Language Models (LLMs). However, rew…