9 papers · 1 filter
Any Time, Any Space: A World-State-Centric Framework for Agentic Video Generation
Weijie Wang, Haomin Zhang, Ye Li +7
Long-form agentic video generation must turn a screenplay into shots that remain spatially and temporally consistent across nonconsecutive requests. Passing prompts or recent frame…
Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
Hanbin Zhou, Shangzhe Li, Alexander Braverman +1
We study adversarial imitation learning (AIL), in which an agent learns to imitate expert demonstrations by optimizing a policy against an adversarial reward that distinguishes exp…
An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning
Shangzhe Li, Yuxiao Yang, Tianrun Yu +4
We study on-policy distillation (OPD) through the lens of reinforcement learning, establishing a connection between the reverse-KL objective in OPD and KL-regularized policy optimi…
Rethinking Training-Inference Mismatch in LLM Reinforcement Learning: Where It Arises and How to Correct It
Tianrun Yu, Kaixiang Zhao, Shangzhe Li +4
We study training-inference mismatch in reinforcement learning with verifiable rewards (RLVR) for large language models, where rollouts are sampled by an inference engine while gra…
When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation
Yuxiao Yang, Tianrun Yu, Shangzhe Li +6
We study length inflation in on-policy distillation (OPD), where student responses can become excessively long and even exhaust the generation budget. We identify \emph{termination…
WatchAct: A Benchmark for Behavior-Grounded Robot Manipulation
Baiqi Li, Ce Zhang, Yu Fang +4
A robot working alongside people must reason about what they have done, in what order, and with what intent. Video carries the spatial layouts, object histories, and gestures that…
Provably Efficient Offline-to-Online Value Adaptation with General Function Approximation
Shangzhe Li, Weitong Zhang
We study value adaptation in offline-to-online reinforcement learning under general function approximation. Starting from an imperfect offline pretrained -function, the learner…
Your Self-Play Algorithm is Secretly an Adversarial Imitator: Understanding LLM Self-Play through the Lens of Imitation Learning
Shangzhe Li, Xuchao Zhang, Chetan Bansal +1
Self-play post-training methods has emerged as an effective approach for finetuning large language models and turn the weak language model into strong language model without prefer…
Imitation from Observations with Trajectory-Level Generative Embeddings
Yongtao Qu, Shangzhe Li, Weitong Zhang
We consider the offline imitation learning from observations (LfO) where the expert demonstrations are scarce and the available offline suboptimal data are far from the expert beha…