2.3k citations · 4.7k across the 84 of their papers we have counts for
98 papers
ContactBench 1.0: A Benchmark for Everyday Contact-Rich Robotic Manipulation
Jingyi Xiang, Yixuan Wang, Jeannette Bohg +1
Contact-rich manipulation underlies many everyday human activities, yet evaluating robots on such tasks remains difficult: the tasks involve diverse contact behaviors and are hard…
ClothAtlas: Joint Garment Identification and State Estimation under Self-Occlusion
Jens Lundell, Alberta Longhini, Lei Shu +6
Garment manipulation requires inferring both garment identity and complete configuration from partial, self-occluded observations. This creates a chicken-and-egg problem: completin…
Global Exponential Convergence of Two-Layer Linear Network Training
Stephen Y Zhang, Gabriel Peyré
We prove global exponential (linear) convergence with an explicit rate in the rich scaling for wide two-layer linear networks trained with smooth Polyak-Lojasiewicz predictor losse…
Deformable Cargo Transport in Microgravity with Astrobee
Daniel Morton, Rika Antonova, Brian Coltin +2
We present pyastrobee: a simulation environment and control stack for Astrobee in Python, with an emphasis on cargo manipulation and transport tasks. We also demonstrate preliminar…
Shadow: Leveraging Segmentation Masks for Cross-Embodiment Policy Transfer
Marion Lepert, Ria Doshi, Jeannette Bohg
Data collection in robotics is spread across diverse hardware, and this variation will increase as new hardware is developed. Effective use of this growing body of data requires me…
Phantom: Training Robots Without Robots Using Only Human Videos
Marion Lepert, Jiaying Fang, Jeannette Bohg
Training general-purpose robots requires learning from large and diverse data sources. Current approaches rely heavily on teleoperated demonstrations which are difficult to scale.…
Force-Aware Autonomous Robotic Surgery
Alaa Eldin Abdelaal, Jiaying Fang, Tim N. Reinhart +4
This work demonstrates the benefits of using tool-tissue interaction forces in the design of autonomous systems in robot-assisted surgery (RAS). Autonomous systems in surgery must…
DexForce: Extracting Force-informed Actions from Kinesthetic Demonstrations for Dexterous Manipulation
Claire Chen, Zhongchun Yu, Hojung Choi +2
Imitation learning requires high-quality demonstrations consisting of sequences of state-action pairs. For contact-rich dexterous manipulation tasks that require dexterity, the act…
TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learning
Jimmy Wu, William Chong, Robert Holmberg +6
Exploiting the promise of recent advances in imitation learning for mobile manipulation will require the collection of large numbers of human-guided demonstrations. This paper prop…
What's the Move? Hybrid Imitation Learning via Salient Points
Priya Sundaresan, Hengyuan Hu, Quan Vuong +2
While imitation learning (IL) offers a promising framework for teaching robots various behaviors, learning complex tasks remains challenging. Existing IL policies struggle to gener…
Get a Grip: Multi-Finger Grasp Evaluation at Scale Enables Robust Sim-to-Real Transfer
Tyler Ga Wei Lum, Albert H. Li, Preston Culbertson +4
This work explores conditions under which multi-finger grasping algorithms can attain robust sim-to-real transfer. While numerous large datasets facilitate learning generative mode…
Neural Attention Field: Emerging Point Relevance in 3D Scenes for One-Shot Dexterous Grasping
Qianxu Wang, Congyue Deng, Tyler Ga Wei Lum +5
One-shot transfer of dexterous grasps to novel scenes with object and context variations has been a challenging problem. While distilled feature fields from large vision models hav…
Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress
Christopher Agia, Rohan Sinha, Jingyun Yang +4
Robot behavior policies trained via imitation learning are prone to failure under conditions that deviate from their training data. Thus, algorithms that monitor learned policies a…
Points2Plans: From Point Clouds to Long-Horizon Plans with Composable Relational Dynamics
Yixuan Huang, Christopher Agia, Jimmy Wu +2
We present Points2Plans, a framework for composable planning with a relational dynamics model that enables robots to solve long-horizon manipulation tasks from partial-view point c…
EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learning
Jingyun Yang, Zi-ang Cao, Congyue Deng +3
Building effective imitation learning methods that enable robots to learn from limited data and still generalize across diverse real-world environments is a long-standing problem i…
Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation
Aaditya Prasad, Kevin Lin, Jimmy Wu +2
Many robotic systems, such as mobile manipulators or quadrotors, cannot be equipped with high-end GPUs due to space, weight, and power constraints. These constraints prevent these…
RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketches
Priya Sundaresan, Quan Vuong, Jiayuan Gu +10
Natural language and images are commonly used as goal representations in goal-conditioned imitation learning (IL). However, natural language can be ambiguous and images can be over…
DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
Alexander Khazatsky, Karl Pertsch, Suraj Nair +98
The creation of large, diverse, high-quality robot manipulation datasets is an important stepping stone on the path toward more capable and robust robotic manipulation policies. Ho…
Tactile-Informed Action Primitives Mitigate Jamming in Dense Clutter
Dane Brouwer, Joshua Citron, Hojung Choi +4
It is difficult for robots to retrieve objects in densely cluttered lateral access scenes with movable objects as jamming against adjacent objects and walls can inhibit progress. W…
Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning
Jingyun Yang, Max Sobol Mark, Brandon Vu +3
The pre-train and fine-tune paradigm in machine learning has had dramatic success in a wide range of domains because the use of existing data or pre-trained models on the internet…
ShaSTA-Fuse: Camera-LiDAR Sensor Fusion to Model Shape and Spatio-Temporal Affinities for 3D Multi-Object Tracking
Tara Sadjadpour, Rares Ambrus, Jeannette Bohg
3D multi-object tracking (MOT) is essential for an autonomous mobile agent to safely navigate a scene. In order to maximize the perception capabilities of the autonomous agent, we…
EquivAct: SIM(3)-Equivariant Visuomotor Policies beyond Rigid Object Manipulation
Jingyun Yang, Congyue Deng, Jimmy Wu +3
If a robot masters folding a kitchen towel, we would expect it to master folding a large beach towel. However, existing policy learning methods that rely on data augmentation still…
AO-Grasp: Articulated Object Grasp Generation
Carlota Parés Morlans, Claire Chen, Yijia Weng +6
We introduce AO-Grasp, a grasp proposal method that generates 6 DoF grasps that enable robots to interact with articulated objects, such as opening and closing cabinets and applian…
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Embodiment Collaboration, Abby O'Neill, Abdul Rehman +291
Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, thi…