38 citations · 141 across the 18 of their papers we have counts for
20 papers
Flex-: A Multi-Stream World-Action Model with Compute Flexibility
Ge Yan, Jinghao Liu, Yuzhi Fan +4
World-action models (WAMs) predict the future to act better, but nearly all of them predict only RGB latents, trained purely for pixel reconstruction, with no explicit signal for t…
TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning
Matthew M. Hong, Jesse Zhang, Anusha Nagabandi +1
Fine-tuning pre-trained robot policies with reinforcement learning (RL) often inherits the bottlenecks introduced by pre-training with behavioral cloning (BC), which produces narro…
Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons
Anthony Liang, Yigit Korkmaz, Jiahui Zhang +14
General-purpose robot reward models are typically trained to predict absolute task progress from expert demonstrations, providing only local, frame-level supervision. While effecti…
PEEK: Guiding and Minimal Image Representations for Zero-Shot Generalization of Robot Manipulation Policies
Jesse Zhang, Marius Memmel, Kevin Kim +6
Robotic manipulation policies often fail to generalize because they must simultaneously learn where to attend, what actions to take, and how to execute them. We argue that high-lev…
HAND Me the Data: Fast Robot Adaptation via Hand Path Retrieval
Matthew Hong, Anthony Liang, Kevin Kim +4
We hand the community HAND, a simple and time-efficient method for teaching robots new manipulation tasks through human hand demonstrations. Instead of relying on task-specific rob…
ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrations
Jiahui Zhang, Yusen Luo, Abrar Anwar +5
We introduce ReWiND, a framework for learning robot manipulation tasks solely from language instructions without per-task demonstrations. Standard reinforcement learning (RL) and i…
HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation
Yi Li, Yuquan Deng, Jesse Zhang +9
Large foundation models have shown strong open-world generalization to complex problems in vision and language, but similar levels of generalization have yet to be achieved in robo…
EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Data
Jesse Zhang, Minho Heo, Zuxin Liu +4
Most reinforcement learning (RL) methods focus on learning optimal policies over low-level action spaces. While these methods can perform well in their training environments, they…
RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback
Yufei Wang, Zhanyi Sun, Jesse Zhang +4
Reward engineering has long been a challenge in Reinforcement Learning (RL) research, as it often requires extensive human effort and iterative processes of trial-and-error to desi…
LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers
Taewook Nam, Juyong Lee, Jesse Zhang +3
We propose a framework that leverages foundation models as teachers, guiding a reinforcement learning agent to acquire semantically meaningful behavior without human feedback. In o…
Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidance
Jesse Zhang, Jiahui Zhang, Karl Pertsch +5
We propose BOSS, an approach that automatically learns to solve new long-horizon, complex, and meaningful tasks by growing a learned skill library with minimal supervision. Prior w…
RoboCLIP: One Demonstration is Enough to Learn Robot Policies
Sumedh A Sontakke, Jesse Zhang, Sébastien M. R. Arnold +5
Reward specification is a notoriously difficult problem in reinforcement learning, requiring extensive expert supervision to design robust reward functions. Imitation learning (IL)…
TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models
Zuxin Liu, Jesse Zhang, Kavosh Asadi +4
The full potential of large pretrained models remains largely untapped in control domains like robotics. This is mainly because of the scarcity of data and the computational challe…
SPRINT: Scalable Policy Pre-Training via Language Instruction Relabeling
Jesse Zhang, Karl Pertsch, Jiahui Zhang +1
Pre-training robot policies with a rich set of skills can substantially accelerate the learning of downstream tasks. Prior works have defined pre-training tasks via natural languag…
Hierarchical Neural Program Synthesis
Linghan Zhong, Ryan Lindeborg, Jesse Zhang +2
Program synthesis aims to automatically construct human-readable programs that satisfy given task specifications, such as input/output pairs or demonstrations. Recent works have de…
Learning to Synthesize Programs as Interpretable and Generalizable Policies
Dweep Trivedi, Jesse Zhang, Shao-Hua Sun +1
Recently, deep reinforcement learning (DRL) methods have achieved impressive performance on tasks in a variety of domains. However, neural network policies produced with DRL method…
Hierarchical Reinforcement Learning By Discovering Intrinsic Options
Jesse Zhang, Haonan Yu, Wei Xu
We propose a hierarchical reinforcement learning method, HIDIO, that can learn task-agnostic options in a self-supervised manner while jointly learning to utilize them to solve spa…
COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning
Avi Singh, Albert Yu, Jonathan Yang +3
Reinforcement learning has been applied to a wide variety of robotics problems, but most of such applications involve collecting data from scratch for each new task. Since the amou…
Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings
Jesse Zhang, Brian Cheung, Chelsea Finn +2
Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous, imperiling the RL agent, other agents, and the environment. To overcome t…
REPLAB: A Reproducible Low-Cost Arm Benchmark Platform for Robotic Learning
Brian Yang, Jesse Zhang, Vitchyr Pong +2
Standardized evaluation measures have aided in the progress of machine learning approaches in disciplines such as computer vision and machine translation. In this paper, we make th…