62 citations · 168 across the 19 of their papers we have counts for
20 papers
GRAIL: Generating Humanoid Loco-Manipulation from World Models
Tianyi Xie, Haotian Zhang, Jinhyung Park +17
Scaling humanoid loco-manipulation requires robot-compatible demonstrations across diverse objects, whole-body motions, and scene geometries, but teleoperation and motion capture a…
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
Bowen Wen, Wei Yang, Jan Kautz +1
We present FoundationPose, a unified foundation model for 6D object pose estimation and tracking, supporting both model-based and model-free setups. Our approach can be instantly a…
Diff-DOPE: Differentiable Deep Object Pose Estimation
Jonathan Tremblay, Bowen Wen, Valts Blukis +3
We introduce Diff-DOPE, a 6-DoF pose refiner that takes as input an image, a 3D textured model of an object, and an initial pose of the object. The method uses differentiable rende…
HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructions
Andrew Guo, Bowen Wen, Jianhe Yuan +4
We present the HANDAL dataset for category-level object pose estimation and affordance prediction. Unlike previous datasets, ours is focused on robotics-ready manipulable objects t…
TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation
Taeyeop Lee, Jonathan Tremblay, Valts Blukis +6
Test-time adaptation methods have been gaining attention recently as a practical solution for addressing source-to-target domain gaps by gradually updating the model without requir…
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
Bowen Wen, Jonathan Tremblay, Valts Blukis +6
We present a near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence, while simultaneously performing neural 3D reconstruction of the obj…
One-Shot Neural Fields for 3D Object Understanding
Valts Blukis, Taeyeop Lee, Jonathan Tremblay +5
We present a unified and compact scene representation for robotics, where each object in the scene is depicted by a latent code capturing geometry and appearance. This representati…
RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control
Zhenggang Tang, Balakumar Sundaralingam, Jonathan Tremblay +6
We present a system for collision-free control of a robot manipulator that uses only RGB views of the world. Perceptual input of a tabletop scene is provided by multiple images of…
Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation
Yunzhi Lin, Thomas Müller, Jonathan Tremblay +5
We present a parallelized optimization method based on fast Neural Radiance Fields (NeRF) for estimating 6-DoF pose of a camera with respect to an object or scene. Given a single o…
Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations
Junchi Liang, Bowen Wen, Kostas Bekris +1
This work aims to learn how to perform complex robot manipulation tasks that are composed of several, consecutively executed low-level sub-tasks, given as input a few visual demons…
Complex In-Hand Manipulation via Compliance-Enabled Finger Gaiting and Multi-Modal Planning
Andrew S. Morgan, Kaiyu Hang, Bowen Wen +2
Constraining contacts to remain fixed on an object during manipulation limits the potential workspace size, as motion is subject to the hand's kinematic topology. Finger gaiting is…
You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration
Bowen Wen, Wenzhao Lian, Kostas Bekris +1
Promising results have been achieved recently in category-level manipulation that generalizes across object instances. Nevertheless, it often requires expensive real-world data col…
CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulation
Bowen Wen, Wenzhao Lian, Kostas Bekris +1
Task-relevant grasping is critical for industrial assembly, where downstream manipulation tasks constrain the set of valid grasps. Learning how to perform this task, however, is ch…
BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Models
Bowen Wen, Kostas Bekris
Tracking the 6D pose of objects in video sequences is important for robot manipulation. Most prior efforts, however, often assume that the target object's CAD model, at least at a…
Vision-driven Compliant Manipulation for Reliable, High-Precision Assembly Tasks
Andrew S. Morgan, Bowen Wen, Junchi Liang +3
Highly constrained manipulation tasks continue to be challenging for autonomous robots as they require high levels of precision, typically less than 1mm, which is often incompatibl…
Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains
Bowen Wen, Chaitanya Mitash, Kostas Bekris
Tracking the 6D pose of objects in video sequences is important for robot manipulation. This work presents se(3)-TrackNet, a data-driven optimization approach for long term, 6D pos…
se(3)-TrackNet: Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains
Bowen Wen, Chaitanya Mitash, Baozhang Ren +1
Tracking the 6D pose of objects in video sequences is important for robot manipulation. This task, however, introduces multiple challenges: (i) robot manipulation involves signific…
Task-driven Perception and Manipulation for Constrained Placement of Unknown Objects
Chaitanya Mitash, Rahul Shome, Bowen Wen +2
Recent progress in robotic manipulation has dealt with the case of previously unknown objects in the context of relatively simple tasks, such as bin-picking. Existing methods for m…
Robust, Occlusion-aware Pose Estimation for Objects Grasped by Adaptive Hands
Bowen Wen, Chaitanya Mitash, Sruthi Soorian +3
Many manipulation tasks, such as placement or within-hand manipulation, require the object's pose relative to a robot hand. The task is difficult when the hand significantly occlud…
Scene-level Pose Estimation for Multiple Instances of Densely Packed Objects
Chaitanya Mitash, Bowen Wen, Kostas Bekris +1
This paper introduces key machine learning operations that allow the realization of robust, joint 6D pose estimation of multiple instances of objects either densely packed or in un…