8 papers
When Does Trajectory-Level Supervision Permit Efficient Offline Reinforcement Learning?
Xuanfei Ren, Tengyang Xie
Offline reinforcement learning is typically analyzed under process-level reward supervision, yet many sequential decision datasets record only trajectory-level outcomes. We develop…
Understanding Behavior Cloning with Action Quantization
Haoqun Cao, Tengyang Xie
Behavior cloning is a fundamental paradigm in machine learning, enabling policy learning from expert demonstrations across robotics, autonomous driving, and generative models. Auto…
Breaking the Capability Ceiling of LLM Post-Training by Reintroducing Markov States
Yurun Yuan, Tengyang Xie
Reinforcement learning (RL) has become a standard paradigm for post-training and aligning Large Language Models (LLMs), yet recent evidence suggests it faces a persistent "capabili…
POLCA: Stochastic Generative Optimization with LLM
Xuanfei Ren, Allen Nie, Tengyang Xie +1
Optimizing complex systems, ranging from LLM prompts to multi-turn agents, traditionally requires labor-intensive manual iteration. We formalize this challenge as a stochastic gene…
Reinforce LLM Reasoning through Multi-Agent Reflection
Yurun Yuan, Tengyang Xie
Leveraging more test-time computation has proven to be an effective way to boost the reasoning capabilities of large language models (LLMs). Among various methods, the verify-and-i…
Outcome-Based Online Reinforcement Learning: Algorithms and Fundamental Limits
Fan Chen, Zeyu Jia, Alexander Rakhlin +1
Reinforcement learning with outcome-based feedback faces a fundamental challenge: when rewards are only observed at trajectory endpoints, how do we assign credit to the right actio…