41 citations · 55 across the 14 of their papers we have counts for
17 papers
Measuring Empowerment from Offline Data for Intrinsically Motivated Exploration
Ishir Garg, Vivek Myers, Qiyang Li +3
Large, unlabeled datasets of prior experience have become increasingly available for agentic tasks in domains such as robotics and coding. A common recipe for these tasks is to tra…
Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latency
Brian Zhu, Momen Khalil, E Harrison +17
While reinforcement learning (RL) allows generalist robot policies to continually improve during deployment, the large model size of modern generalist policies, such as VLAs, poses…
Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning
Zhiyuan Zhou, Andy Peng, Charles Xu +4
Expressive continuous control policies, such as diffusion and flow models, form the backbone of recent advances in scaling imitation learning for simulated and real robot control.…
Q-learning with Adjoint Matching
Qiyang Li, Sergey Levine
We propose Q-learning with Adjoint Matching (QAM), a novel TD-based reinforcement learning (RL) algorithm that tackles a long-standing challenge in continuous-action RL: efficient…
Decoupled Q-Chunking
Qiyang Li, Seohong Park, Sergey Levine
Temporal-difference (TD) methods learn state and action values efficiently by bootstrapping from their own future value predictions, but such a self-bootstrapping mechanism is pron…
EXPO: Stable Reinforcement Learning with Expressive Policies
Perry Dong, Qiyang Li, Dorsa Sadigh +1
We study the problem of training and fine-tuning expressive policies with online reinforcement learning (RL) given an offline dataset. Training expressive policy classes with onlin…
Reinforcement Learning with Action Chunking
Qiyang Li, Zhiyuan Zhou, Sergey Levine
We present Q-chunking, a simple yet effective recipe for improving reinforcement learning (RL) algorithms for long-horizon, sparse-reward tasks. Our recipe is designed for the offl…
Flow Q-Learning
Seohong Park, Qiyang Li, Sergey Levine
We present flow Q-learning (FQL), a simple and performant offline reinforcement learning (RL) method that leverages an expressive flow-matching policy to model arbitrarily complex…
RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning
Charles Xu, Qiyang Li, Jianlan Luo +1
Recent advances in robotic foundation models have enabled the development of generalist policies that can adapt to diverse tasks. While these models show impressive flexibility, th…
Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data
Zhiyuan Zhou, Andy Peng, Qiyang Li +2
The modern paradigm in machine learning involves pre-training on diverse data, followed by task-specific fine-tuning. In reinforcement learning (RL), this translates to learning vi…
Leveraging Skills from Unlabeled Prior Data for Efficient Online Exploration
Max Wilcoxson, Qiyang Li, Kevin Frans +1
Unsupervised pretraining has been transformative in many supervised domains. However, applying such ideas to reinforcement learning (RL) presents a unique challenge in that fine-tu…
Learning Visuotactile Skills with Two Multifingered Hands
Toru Lin, Yu Zhang, Qiyang Li +4
Aiming to replicate human-like dexterity, perceptual experiences, and motion patterns, we explore learning from human demonstrations using a bimanual system with multifingered hand…
Accelerating Exploration with Unlabeled Prior Data
Qiyang Li, Jason Zhang, Dibya Ghosh +2
Learning to solve tasks from a sparse reward signal is a major challenge for standard reinforcement learning (RL) algorithms. However, in the real world, agents rarely need to solv…
Efficient Deep Reinforcement Learning Requires Regulating Overfitting
Qiyang Li, Aviral Kumar, Ilya Kostrikov +1
Deep reinforcement learning algorithms that learn policies by trial-and-error must learn from limited amounts of data collected by actively interacting with the environment. While…
Understanding the Complexity Gains of Single-Task RL with a Curriculum
Qiyang Li, Yuexiang Zhai, Yi Ma +1
Reinforcement learning (RL) problems can be challenging without well-shaped rewards. Prior work on provably efficient RL methods generally proposes to address this issue with dedic…
AdaCat: Adaptive Categorical Discretization for Autoregressive Models
Qiyang Li, Ajay Jain, Pieter Abbeel
Autoregressive generative models can estimate complex continuous data distributions, like trajectory rollouts in an RL environment, image intensities, and audio. Most state-of-the-…
Offline Reinforcement Learning as One Big Sequence Modeling Problem
Michael Janner, Qiyang Li, Sergey Levine
Reinforcement learning (RL) is typically concerned with estimating stationary policies or single-step models, leveraging the Markov property to factorize problems in time. However,…