2.3k citations · 3.8k across the 166 of their papers we have counts for
34 papers · 1 filter
Chain of Code: Reasoning with a Language Model-Augmented Code Emulator
Chengshu Li, Jacky Liang, Andy Zeng +7
Code provides a general syntactic structure to build complex programs and perform precise computations when paired with a code interpreter - we hypothesize that language models (LM…
Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections
Lihan Zha, Yuchen Cui, Li-Heng Lin +5
Today's robot policies exhibit subpar performance when faced with the challenge of generalizing to novel environments. Human corrective feedback is a crucial form of guidance to en…
Imitation Bootstrapped Reinforcement Learning
Hengyuan Hu, Suvir Mirchandani, Dorsa Sadigh
Despite the considerable potential of reinforcement learning (RL), robotic control tasks predominantly rely on imitation learning (IL) due to its better sample efficiency. However,…
Stabilize to Act: Learning to Coordinate for Bimanual Manipulation
Jennifer Grannen, Yilin Wu, Brandon Vu +1
Key to rich, dexterous manipulation in the real world is the ability to coordinate control across two hands. However, while the promise afforded by bimanual robotic systems is imme…
Diverse Conventions for Human-AI Collaboration
Bidipta Sarkar, Andy Shih, Dorsa Sadigh
Conventions are crucial for strong performance in cooperative multi-agent games, because they allow players to coordinate on a shared strategy without explicit communication. Unfor…
Learning Sequential Acquisition Policies for Robot-Assisted Feeding
Priya Sundaresan, Jiajun Wu, Dorsa Sadigh
A robot providing mealtime assistance must perform specialized maneuvers with various utensils in order to pick up and feed a range of food items. Beyond these dexterous low-level…
RoboCLIP: One Demonstration is Enough to Learn Robot Policies
Sumedh A Sontakke, Jesse Zhang, Sébastien M. R. Arnold +5
Reward specification is a notoriously difficult problem in reinforcement learning, requiring extensive expert supervision to design robust reward functions. Imitation learning (IL)…
Contrastive Preference Learning: Learning from Human Feedback without RL
Joey Hejna, Rafael Rafailov, Harshit Sikchi +4
Reinforcement Learning from Human Feedback (RLHF) has emerged as a popular paradigm for aligning models with human intent. Typically RLHF algorithms operate in two phases: first, u…
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Embodiment Collaboration, Abby O'Neill, Abdul Rehman +291
Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, thi…
Robots That Can See: Leveraging Human Pose for Trajectory Prediction
Tim Salzmann, Lewis Chiang, Markus Ryll +3
Anticipating the motion of all humans in dynamic environments such as homes and offices is critical to enable safe and effective robot navigation. Such spaces remain challenging as…
Gesture-Informed Robot Assistance via Foundation Models
Li-Heng Lin, Yuchen Cui, Yilun Hao +2
Gestures serve as a fundamental and significant mode of non-verbal communication among humans. Deictic gestures (such as pointing towards an object), in particular, offer valuable…
Physically Grounded Vision-Language Models for Robotic Manipulation
Jensen Gao, Bidipta Sarkar, Fei Xia +5
Recent advances in vision-language models (VLMs) have led to improved performance on tasks such as visual question answering and image captioning. Consequently, these models are no…
Polybot: Training One Policy Across Robots While Embracing Variability
Jonathan Yang, Dorsa Sadigh, Chelsea Finn
Reusing large datasets is crucial to scale vision-based robotic manipulators to everyday scenarios due to the high cost of collecting robotic datasets. However, robotic platforms p…
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Stephen Casper, Xander Davies, Claudia Shi +29
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals. RLHF has emerged as the central method used to finetune state-of…
Large Language Models as General Pattern Machines
Suvir Mirchandani, Fei Xia, Pete Florence +6
We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabi…
Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners
Allen Z. Ren, Anushri Dixit, Alexandra Bodrova +11
Large language models (LLMs) exhibit a wide range of promising capabilities -- from step-by-step planning to commonsense reasoning -- that may provide utility for robots, but remai…
Language to Rewards for Robotic Skill Synthesis
Wenhao Yu, Nimrod Gileadi, Chuyuan Fu +17
Large language models (LLMs) have demonstrated exciting progress in acquiring diverse new capabilities through in-context learning, ranging from logical reasoning to code-writing.…
Generating Language Corrections for Teaching Physical Control Tasks
Megha Srivastava, Noah Goodman, Dorsa Sadigh
AI assistance continues to help advance applications in education, from language learning to intelligent tutoring systems, yet current methods for providing students feedback are s…
Data Quality in Imitation Learning
Suneel Belkhale, Yuchen Cui, Dorsa Sadigh
In supervised learning, the question of data quality and curation has been over-shadowed in recent years by increasingly more powerful and expressive models that can ingest interne…
HYDRA: Hybrid Robot Actions for Imitation Learning
Suneel Belkhale, Yuchen Cui, Dorsa Sadigh
Imitation Learning (IL) is a sample efficient paradigm for robot learning using expert demonstrations. However, policies learned through IL suffer from state distribution shift at…
KITE: Keypoint-Conditioned Policies for Semantic Manipulation
Priya Sundaresan, Suneel Belkhale, Dorsa Sadigh +1
While natural language offers a convenient shared interface for humans and robots, enabling robots to interpret and follow language commands remains a longstanding challenge in man…
Toward Grounded Commonsense Reasoning
Minae Kwon, Hengyuan Hu, Vivek Myers +3
Consider a robot tasked with tidying a desk with a meticulously constructed Lego sports car. A human may recognize that it is not appropriate to disassemble the sports car and put…
Strategic Reasoning with Language Models
Kanishk Gandhi, Dorsa Sadigh, Noah D. Goodman
Strategic reasoning enables agents to cooperate, communicate, and compete with other agents in diverse situations. Existing approaches to solving strategic games rely on extensive…
Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets
Maximilian Du, Suraj Nair, Dorsa Sadigh +1
Enabling robots to learn novel visuomotor skills in a data-efficient manner remains an unsolved problem with myriad challenges. A popular paradigm for tackling this problem is thro…