163 citations · 420 across the 27 of their papers we have counts for
28 papers
SPRINT: Scalable Policy Pre-Training via Language Instruction Relabeling
Jesse Zhang, Karl Pertsch, Jiahui Zhang +1
Pre-training robot policies with a rich set of skills can substantially accelerate the learning of downstream tasks. Prior works have defined pre-training tasks via natural languag…
FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulation
Minho Heo, Youngwoon Lee, Doohyun Lee +1
Reinforcement learning (RL), imitation learning (IL), and task and motion planning (TAMP) have demonstrated impressive performance across various robotic manipulation tasks. Howeve…
Hierarchical Neural Program Synthesis
Linghan Zhong, Ryan Lindeborg, Jesse Zhang +2
Program synthesis aims to automatically construct human-readable programs that satisfy given task specifications, such as input/output pairs or demonstrations. Recent works have de…
QMP: Q-switch Mixture of Policies for Multi-Task Behavior Sharing
Grace Zhang, Ayush Jain, Injune Hwang +2
Multi-task reinforcement learning (MTRL) aims to learn several tasks simultaneously for better sample efficiency than learning them separately. Traditional methods achieve this by…
Cross-Domain Transfer via Semantic Skill Imitation
Karl Pertsch, Ruta Desai, Vikash Kumar +4
We propose an approach for semantic imitation, which uses demonstrations from a source domain, e.g. human videos, to accelerate reinforcement learning (RL) in a different target do…
PATO: Policy Assisted TeleOperation for Scalable Robot Data Collection
Shivin Dass, Karl Pertsch, Hejia Zhang +3
Large-scale data is an essential component of machine learning as demonstrated in recent advances in natural language processing and computer vision research. However, collecting l…
Skill-based Model-based Reinforcement Learning
Lucy Xiaoyang Shi, Joseph J. Lim, Youngwoon Lee
Model-based reinforcement learning (RL) is a sample-efficient way of learning complex behaviors by leveraging a learned single-step dynamics model to plan actions in imagination. H…
Skill-based Meta-Reinforcement Learning
Taewook Nam, Shao-Hua Sun, Karl Pertsch +2
While deep reinforcement learning methods have shown impressive results in robot learning, their sample inefficiency makes the learning of complex, long-horizon behaviors with real…
Task-Induced Representation Learning
Jun Yamada, Karl Pertsch, Anisha Gunjal +1
In this work, we evaluate the effectiveness of representation learning approaches for decision making in visually complex environments. Representation learning is essential for eff…
Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization
Youngwoon Lee, Joseph J. Lim, Anima Anandkumar +1
Skill chaining is a promising approach for synthesizing complex behaviors by sequentially combining previously learned skills. Yet, a naive composition of skills fails when a polic…
Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation
I-Chun Arthur Liu, Shagun Uppal, Gaurav S. Sukhatme +3
Learning complex manipulation tasks in realistic, obstructed environments is a challenging problem due to hard exploration in the presence of obstacles and high-dimensional visual…
Learning to Synthesize Programs as Interpretable and Generalizable Policies
Dweep Trivedi, Jesse Zhang, Shao-Hua Sun +1
Recently, deep reinforcement learning (DRL) methods have achieved impressive performance on tasks in a variety of domains. However, neural network policies produced with DRL method…
Demonstration-Guided Reinforcement Learning with Learned Skills
Karl Pertsch, Youngwoon Lee, Yue Wu +1
Demonstration-guided reinforcement learning (RL) is a promising approach for learning complex behaviors by leveraging both reward feedback and a set of target task demonstrations.…
Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding
Grace Zhang, Linghan Zhong, Youngwoon Lee +1
The ability to transfer a policy from one environment to another is a promising avenue for efficient robot learning in realistic settings where task supervision is not available. T…
Generalization to New Actions in Reinforcement Learning
Ayush Jain, Andrew Szot, Joseph J. Lim
A fundamental trait of intelligence is the ability to achieve goals in the face of novel circumstances, such as making decisions from new action choices. However, standard reinforc…
Accelerating Reinforcement Learning with Learned Skill Priors
Karl Pertsch, Youngwoon Lee, Joseph J. Lim
Intelligent agents rely heavily on prior experience when learning a new task, yet most modern reinforcement learning (RL) approaches learn every task from scratch. One approach for…
Motion Planner Augmented Reinforcement Learning for Robot Manipulation in Obstructed Environments
Jun Yamada, Youngwoon Lee, Gautam Salhotra +5
Deep reinforcement learning (RL) agents are able to learn contact-rich manipulation tasks by maximizing a reward signal, but require large amounts of experience, especially in envi…
To Follow or not to Follow: Selective Imitation Learning from Observations
Youngwoon Lee, Edward S. Hu, Zhengyu Yang +1
Learning from demonstrations is a useful way to transfer a skill from one agent to another. While most imitation learning methods aim to mimic an expert skill by following the demo…
IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks
Youngwoon Lee, Edward S. Hu, Zhengyu Yang +2
The IKEA Furniture Assembly Environment is one of the first benchmarks for testing and accelerating the automation of complex manipulation tasks. The environment is designed to adv…
Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation
Risto Vuorio, Shao-Hua Sun, Hexiang Hu +1
Model-agnostic meta-learners aim to acquire meta-learned parameters from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. With the flexib…
Keyframing the Future: Keyframe Discovery for Visual Prediction and Planning
Karl Pertsch, Oleh Rybkin, Jingyun Yang +5
Temporal observations such as videos contain essential information about the dynamics of the underlying scene, but they are often interleaved with inessential, predictable details.…
Toward Multimodal Model-Agnostic Meta-Learning
Risto Vuorio, Shao-Hua Sun, Hexiang Hu +1
Gradient-based meta-learners such as MAML are able to learn a meta-prior from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. One import…
Simulator Predictive Control: Using Learned Task Representations and MPC for Zero-Shot Generalization and Sequencing
Zhanpeng He, Ryan Julian, Eric Heiden +5
Simulation-to-real transfer is an important strategy for making reinforcement learning practical with real robots. Successful sim-to-real transfer systems have difficulty producing…
Auto-conditioned Recurrent Mixture Density Networks for Learning Generalizable Robot Skills
Hejia Zhang, Eric Heiden, Stefanos Nikolaidis +2
Personal robots assisting humans must perform complex manipulation tasks that are typically difficult to specify in traditional motion planning pipelines, where multiple objectives…