262 citations · 430 across the 32 of their papers we have counts for
37 papers
GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks
Kaiyuan Chen, Shuangyu Xie, Letian Fu +21
For robots to work reliably in commercial and industrial applications, can recent advances in agentic coding systems combine interpretable robot programming with the open-world ada…
GraspGen-X: Cross-Embodiment 6-DOF Diffusion-based Grasping
Beining Han, Yu-Wei Chao, Erwin Coumans +5
We study cross-embodiment 6-DOF robot grasping. Unlike prior works, we require the model not only to generalize to novel objects / scenes but also to novel gripper morphologies and…
cuRoboV2: Dynamics-Aware Motion Generation with Depth-Fused Distance Fields for High-DoF Robots
Balakumar Sundaralingam, Adithyavairavan Murali, Stan Birchfield
Effective robot autonomy requires motion generation that is safe, feasible, and reactive. Current methods are fragmented: fast planners output physically unexecutable trajectories,…
VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning
Binghao Huang, Jie Xu, Iretiayo Akinola +8
Humans excel at bimanual assembly tasks by adapting to rich tactile feedback -- a capability that remains difficult to replicate in robots through behavioral cloning alone, due to…
Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control
Jun Yamada, Adithyavairavan Murali, Ajay Mandlekar +3
Grasping of diverse objects in unstructured environments remains a significant challenge. Open-loop grasping methods, effective in controlled settings, struggle in cluttered enviro…
GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Training
Adithyavairavan Murali, Balakumar Sundaralingam, Yu-Wei Chao +7
Grasping is a fundamental robot skill, yet despite significant research advancements, learning-based 6-DOF grasping approaches are still not turnkey and struggle to generalize acro…
Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learning
Neel Jawale, Byron Boots, Balakumar Sundaralingam +1
We investigate the problem of teaching a robot manipulator to perform dynamic non-prehensile object transport, also known as the `robot waiter' task, from a limited set of real-wor…
Inference-Time Policy Steering through Human Interactions
Yanwei Wang, Lirui Wang, Yilun Du +6
Generative policies trained with human demonstrations can autonomously accomplish multimodal, long-horizon tasks. However, during inference, humans are often removed from the polic…
DiffusionSeeder: Seeding Motion Optimization with Diffusion for Rapid Motion Planning
Huang Huang, Balakumar Sundaralingam, Arsalan Mousavian +3
Running optimization across many parallel seeds leveraging GPU compute have relaxed the need for a good initialization, but this can fail if the problem is highly non-convex as all…
cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation
Balakumar Sundaralingam, Siva Kumar Sastry Hari, Adam Fishman +9
This paper explores the problem of collision-free motion generation for manipulators by formulating it as a global motion optimization problem. We develop a parallel optimization t…
VaPr: Variable-Precision Tensors to Accelerate Robot Motion Planning
Yu-Shun Hsiao, Siva Kumar Sastry Hari, Balakumar Sundaralingam +5
High-dimensional motion generation requires numerical precision for smooth, collision-free solutions. Typically, double-precision or single-precision floating-point (FP) formats ar…
Diff-DOPE: Differentiable Deep Object Pose Estimation
Jonathan Tremblay, Bowen Wen, Valts Blukis +3
We introduce Diff-DOPE, a 6-DoF pose refiner that takes as input an image, a 3D textured model of an object, and an initial pose of the object. The method uses differentiable rende…
Meta-Policy Learning over Plan Ensembles for Robust Articulated Object Manipulation
Constantinos Chamzas, Caelan Garrett, Balakumar Sundaralingam +2
Recent work has shown that complex manipulation skills, such as pushing or pouring, can be learned through state-of-the-art learning based techniques, such as Reinforcement Learnin…
DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality
Ankur Handa, Arthur Allshire, Viktor Makoviychuk +11
Recent work has demonstrated the ability of deep reinforcement learning (RL) algorithms to learn complex robotic behaviours in simulation, including in the domain of multi-fingered…
RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control
Zhenggang Tang, Balakumar Sundaralingam, Jonathan Tremblay +6
We present a system for collision-free control of a robot manipulator that uses only RGB views of the world. Perceptual input of a tabletop scene is provided by multiple images of…
Neural Motion Fields: Encoding Grasp Trajectories as Implicit Value Functions
Yun-Chun Chen, Adithyavairavan Murali, Balakumar Sundaralingam +3
The pipeline of current robotic pick-and-place methods typically consists of several stages: grasp pose detection, finding inverse kinematic solutions for the detected poses, plann…
HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers
Yu-Wei Chao, Chris Paxton, Yu Xiang +6
We introduce a new simulation benchmark "HandoverSim" for human-to-robot object handovers. To simulate the giver's motion, we leverage a recent motion capture dataset of hand grasp…
Correcting Robot Plans with Natural Language Feedback
Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis +5
When humans design cost or goal specifications for robots, they often produce specifications that are ambiguous, underspecified, or beyond planners' ability to solve. In these case…
Model Predictive Control for Fluid Human-to-Robot Handovers
Wei Yang, Balakumar Sundaralingam, Chris Paxton +4
Human-robot handover is a fundamental yet challenging task in human-robot interaction and collaboration. Recently, remarkable progressions have been made in human-to-robot handover…
DefGraspSim: Physics-based simulation of grasp outcomes for 3D deformable objects
Isabella Huang, Yashraj Narang, Clemens Eppner +5
Robotic grasping of 3D deformable objects (e.g., fruits/vegetables, internal organs, bottles/boxes) is critical for real-world applications such as food processing, robotic surgery…
Learning Perceptual Concepts by Bootstrapping from Human Queries
Andreea Bobu, Chris Paxton, Wei Yang +4
When robots operate in human environments, it's critical that humans can quickly teach them new concepts: object-centric properties of the environment that they care about (e.g. ob…
Geometric Fabrics: Generalizing Classical Mechanics to Capture the Physics of Behavior
Karl Van Wyk, Mandy Xie, Anqi Li +9
Classical mechanical systems are central to controller design in energy shaping methods of geometric control. However, their expressivity is limited by position-only metrics and th…
DefGraspSim: Simulation-based grasping of 3D deformable objects
Isabella Huang, Yashraj Narang, Clemens Eppner +4
Robotic grasping of 3D deformable objects (e.g., fruits/vegetables, internal organs, bottles/boxes) is critical for real-world applications such as food processing, robotic surgery…
STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation
Mohak Bhardwaj, Balakumar Sundaralingam, Arsalan Mousavian +4
Sampling-based model-predictive control (MPC) is a promising tool for feedback control of robots with complex, non-smooth dynamics, and cost functions. However, the computationally…