24 citations · 60 across the 25 of their papers we have counts for
30 papers
Generalizable Robotic Insertion with World Models
Nicklas Hansen, Iretiayo Akinola, Yijie Guo +7
Robotic assembly in high-mixture settings requires adaptable systems that can handle diverse parts, yet current approaches typically rely on policies specialized to each insertion…
SILO: Simulation-in-the-Loop Sim-to-Real Transfer for Multi-Stage Cable Routing
Stone Tao, Jie Xu, Hesam Rabeti +3
Linear-deformable manipulation remains challenging due to the complex deformations of objects such as cables and ropes. Prior data-driven approaches, particularly imitation learnin…
SPARR: Simulation-based Policies with Asymmetric Real-world Residuals for Assembly
Yijie Guo, Iretiayo Akinola, Lars Johannsmeier +3
Robotic assembly presents a long-standing challenge due to its requirement for precise, contact-rich manipulation. While simulation-based learning has enabled the development of ro…
Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning
NVIDIA, :, Mayank Mittal +104
We present Isaac Lab, the natural successor to Isaac Gym, which extends the paradigm of GPU-native robotics simulation into the era of large-scale multi-modal learning. Isaac Lab c…
The Reality Gap in Robotics: Challenges, Solutions, and Best Practices
Elie Aljalbout, Jiaxu Xing, Angel Romero +9
Machine learning has facilitated significant advancements across various robotics domains, including navigation, locomotion, and manipulation. Many such achievements have been driv…
VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning
Binghao Huang, Jie Xu, Iretiayo Akinola +8
Humans excel at bimanual assembly tasks by adapting to rich tactile feedback -- a capability that remains difficult to replicate in robots through behavioral cloning alone, due to…
Refinery: Active Fine-tuning and Deployment-time Optimization for Contact-Rich Policies
Bingjie Tang, Iretiayo Akinola, Jie Xu +6
Simulation-based learning has enabled policies for precise, contact-rich tasks (e.g., robotic assembly) to reach high success rates (~80%) under high levels of observation noise an…
Neural Robot Dynamics
Jie Xu, Eric Heiden, Iretiayo Akinola +3
Accurate and efficient simulation of modern robots remains challenging due to their high degrees of freedom and intricate mechanisms. Neural simulators have emerged as a promising…
MatchMaker: Automated Asset Generation for Robotic Assembly
Yian Wang, Bingjie Tang, Chuang Gan +4
Robotic assembly remains a significant challenge due to complexities in visual perception, functional grasping, contact-rich manipulation, and performing high-precision tasks. Simu…
SRSA: Skill Retrieval and Adaptation for Robotic Assembly Tasks
Yijie Guo, Bingjie Tang, Iretiayo Akinola +3
Enabling robots to learn novel tasks in a data-efficient manner is a long-standing challenge. Common strategies involve carefully leveraging prior experiences, especially transitio…
TacSL: A Library for Visuotactile Sensor Simulation and Learning
Iretiayo Akinola, Jie Xu, Jan Carius +2
For both humans and robots, the sense of touch, known as tactile sensing, is critical for performing contact-rich manipulation tasks. Three key challenges in robotic tactile sensin…
FORGE: Force-Guided Exploration for Robust Contact-Rich Manipulation under Uncertainty
Michael Noseworthy, Bingjie Tang, Bowen Wen +7
We present FORGE, a method for sim-to-real transfer of force-aware manipulation policies in the presence of significant pose uncertainty. During simulation-based policy learning, F…
AutoMate: Specialist and Generalist Assembly Policies over Diverse Geometries
Bingjie Tang, Iretiayo Akinola, Jie Xu +7
Robotic assembly for high-mixture settings requires adaptivity to diverse parts and poses, which is an open challenge. Meanwhile, in other areas of robotics, large models and sim-t…
Signatures Meet Dynamic Programming: Generalizing Bellman Equations for Trajectory Following
Motoya Ohnishi, Iretiayo Akinola, Jie Xu +2
Path signatures have been proposed as a powerful representation of paths that efficiently captures the path's analytic and geometric characteristics, having useful algebraic proper…
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
Ajay Mandlekar, Soroush Nasiriany, Bowen Wen +5
Imitation learning from a large set of human demonstrations has proved to be an effective paradigm for building capable robot agents. However, the demonstrations can be extremely c…
Learning to Summarize and Answer Questions about a Virtual Robot's Past Actions
Chad DeChant, Iretiayo Akinola, Daniel Bauer
When robots perform long action sequences, users will want to easily and reliably find out what they have done. We therefore demonstrate the task of learning to summarize and answe…
IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality
Bingjie Tang, Michael A. Lin, Iretiayo Akinola +5
Robotic assembly is a longstanding challenge, requiring contact-rich interaction and high precision and accuracy. Many applications also require adaptivity to diverse parts, poses,…
Factory: Fast Contact for Robotic Assembly
Yashraj Narang, Kier Storey, Iretiayo Akinola +9
Robotic assembly is one of the oldest and most challenging applications of robotics. In other areas of robotics, such as perception and grasping, simulation has rapidly accelerated…
Model Predictive Control for Fluid Human-to-Robot Handovers
Wei Yang, Balakumar Sundaralingam, Chris Paxton +4
Human-robot handover is a fundamental yet challenging task in human-robot interaction and collaboration. Recently, remarkable progressions have been made in human-to-robot handover…
Assistive Tele-op: Leveraging Transformers to Collect Robotic Task Demonstrations
Henry M. Clever, Ankur Handa, Hammad Mazhar +7
Sharing autonomy between robots and human operators could facilitate data collection of robotic task demonstrations to continuously improve learned models. Yet, the means to commun…
Geometric Fabrics: Generalizing Classical Mechanics to Capture the Physics of Behavior
Karl Van Wyk, Mandy Xie, Anqi Li +9
Classical mechanical systems are central to controller design in energy shaping methods of geometric control. However, their expressivity is limited by position-only metrics and th…
Visionary: Vision architecture discovery for robot learning
Iretiayo Akinola, Anelia Angelova, Yao Lu +5
We propose a vision-based architecture search algorithm for robot manipulation learning, which discovers interactions between low dimension action inputs and high dimensional visua…
CLAMGen: Closed-Loop Arm Motion Generation via Multi-view Vision-Based RL
Iretiayo Akinola, Zizhao Wang, Peter Allen
We propose a vision-based reinforcement learning (RL) approach for closed-loop trajectory generation in an arm reaching problem. Arm trajectory generation is a fundamental robotics…
Dynamic Grasping with Reachability and Motion Awareness
Iretiayo Akinola, Jingxi Xu, Shuran Song +1
Grasping in dynamic environments presents a unique set of challenges. A stable and reachable grasp can become unreachable and unstable as the target object moves, motion planning n…