26 citations · 63 across the 24 of their papers we have counts for
25 papers
Provably Efficient Policy-Reward Co-Pretraining for Adversarial Imitation Learning
Tian Xu, Zexuan Chen, Zhilong Zhang +4
Adversarial imitation learning (AIL) achieves high-quality imitation compared to behavioral cloning (BC), but demands substantial online environment interaction. Recent empirical w…
Cooperative Long Rope Skipping via Multi-Agent Reinforcement Learning
Zihao Wang, Shijie Peng, Kerui Wu +6
Humans exhibit remarkable motor agility, enabling a wide range of dynamic skills such as running and jumping, which highlights the great potential of humanoid robots for athletic l…
How Can Reinforcement Learning Achieve Expert-level Placement?
Ruo-Tong Chen, Ke Xue, Chengrui Gao +7
Chip placement is a critical step in physical design. While reinforcement learning (RL)-based methods have recently emerged, their training primarily focuses on wirelength optimiza…
Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms
Tian Xu, Zhilong Zhang, Zexuan Chen +3
Adversarial imitation learning (AIL), a prominent approach in imitation learning, has achieved significant practical success powered by neural network approximation. However, exist…
Off-Policy Value-Based Reinforcement Learning for Large Language Models
Peng-Yuan Wang, Ziniu Li, Tian Xu +8
Improving data utilization efficiency is critical for scaling reinforcement learning (RL) for long-horizon tasks where generating trajectories is expensive. However, the dominant R…
Non-Adversarial Imitation Learning Provably Free of Compounding Errors: The Value Flow Mechanism
Tian Xu, Chenyang Wang, Xiaochen Zhai +3
Adversarial imitation learning (AIL) achieves high-quality imitation by mitigating compounding errors inherent to behavioral cloning (BC), yet its adversarial optimization frequent…
Eval-Actions: Fine-Grained Execution Quality Evaluation for Robotic Manipulation
Mengyuan Liu, Juyi Sheng, Peiming Li +4
Although Vision--Action (VA) and Vision--Language--Action (VLA) policies have advanced robotic manipulation, their evaluation remains dominated by binary success rates, which obscu…
Generalist Reward Models: Found Inside Large Language Models
Yi-Chen Li, Tian Xu, Yang Yu +6
The alignment of Large Language Models (LLMs) is critically dependent on reward models trained on costly human preference data. While recent work explores bypassing this cost with…
RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning
Charles Xu, Qiyang Li, Jianlan Luo +1
Recent advances in robotic foundation models have enabled the development of generalist policies that can adapt to diverse tasks. While these models show impressive flexibility, th…
Provably and Practically Efficient Adversarial Imitation Learning with General Function Approximation
Tian Xu, Zhilong Zhang, Ruishuo Chen +2
As a prominent category of imitation learning methods, adversarial imitation learning (AIL) has garnered significant practical success powered by neural network approximation. Howe…
Preserving Diversity in Supervised Fine-Tuning of Large Language Models
Ziniu Li, Congliang Chen, Tian Xu +4
Large Language Models (LLMs) typically rely on Supervised Fine-Tuning (SFT) to specialize in downstream tasks, with the Cross Entropy (CE) loss being the de facto choice. However,…
Policy Optimization in RLHF: The Impact of Out-of-preference Data
Ziniu Li, Tian Xu, Yang Yu
Aligning intelligent agents with human preferences and values is important. This paper examines two popular alignment methods: Direct Preference Optimization (DPO) and Reward-Model…
Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learning
Fan-Ming Luo, Tian Xu, Xingchen Cao +1
Learning a precise dynamics model can be crucial for offline reinforcement learning, which, unfortunately, has been found to be quite challenging. Dynamics models that are learned…
ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models
Ziniu Li, Tian Xu, Yushun Zhang +4
Reinforcement Learning from Human Feedback (RLHF) is key to aligning Large Language Models (LLMs), typically paired with the Proximal Policy Optimization (PPO) algorithm. While PPO…
Provably Efficient Adversarial Imitation Learning with Unknown Transitions
Tian Xu, Ziniu Li, Yang Yu +1
Imitation learning (IL) has proven to be an effective method for learning good policies from expert demonstrations. Adversarial imitation learning (AIL), a subset of IL methods, is…
Theoretical Analysis of Offline Imitation With Supplementary Dataset
Ziniu Li, Tian Xu, Yang Yu +1
Behavioral cloning (BC) can recover a good policy from abundant expert data, but may fail when expert data is insufficient. This paper considers a situation where, besides the smal…
Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysis
Tian Xu, Ziniu Li, Yang Yu +1
Imitation learning learns a policy from expert trajectories. While the expert data is believed to be crucial for imitation quality, it was found that a kind of imitation learning a…
A Survey on Model-based Reinforcement Learning
Fan-Ming Luo, Tian Xu, Hang Lai +3
Reinforcement learning (RL) solves sequential decision-making problems via a trial-and-error process interacting with the environment. While RL achieves outstanding success in play…
Model Generation with Provable Coverability for Offline Reinforcement Learning
Chengxing Jia, Hao Yin, Chenxiao Gao +4
Model-based offline optimization with dynamics-aware policy provides a new perspective for policy learning and out-of-distribution generalization, where the learned policy could ad…
A Note on Target Q-learning For Solving Finite MDPs with A Generative Oracle
Ziniu Li, Tian Xu, Yang Yu
Q-learning with function approximation could diverge in the off-policy setting and the target network is a powerful technique to address this issue. In this manuscript, we examine…
Rethinking ValueDice: Does It Really Improve Performance?
Ziniu Li, Tian Xu, Yang Yu +1
Since the introduction of GAIL, adversarial imitation learning (AIL) methods attract lots of research interests. Among these methods, ValueDice has achieved significant improvement…
On Generalization of Adversarial Imitation Learning and Beyond
Tian Xu, Ziniu Li, Yang Yu +1
Despite massive empirical evaluations, one of the fundamental questions in imitation learning is still not fully settled: does AIL (adversarial imitation learning) provably general…
Reinforcement Learning With Sparse-Executing Actions via Sparsity Regularization
Jing-Cheng Pang, Tian Xu, Shengyi Jiang +2
Reinforcement learning (RL) has demonstrated impressive performance in decision-making tasks like embodied control, autonomous driving and financial trading. In many decision-makin…
Error Bounds of Imitating Policies and Environments
Tian Xu, Ziniu Li, Yang Yu
Imitation learning trains a policy by mimicking expert demonstrations. Various imitation methods were proposed and empirically evaluated, meanwhile, their theoretical understanding…