262 citations · 881 across the 33 of their papers we have counts for
34 papers
ManipulationNet: An Infrastructure for Benchmarking Real-World Robot Manipulation with Physical Skill Challenges and Embodied Multimodal Reasoning
Yiting Chen, Kenneth Kimble, Edward H. Adelson +20
Dexterous manipulation enables robots to purposefully alter the physical world, transforming them from passive observers into active agents in unstructured environments. This capab…
SCENEREPLICA: Benchmarking Real-World Robot Manipulation by Creating Replicable Scenes
Ninad Khargonkar, Sai Haneesh Allu, Yangxiao Lu +3
We present a new reproducible benchmark for evaluating robot manipulation in the real world, specifically focusing on pick-and-place. Our benchmark uses the YCB objects, a commonly…
Self-Supervised Unseen Object Instance Segmentation via Long-Term Robot Interaction
Yangxiao Lu, Ninad Khargonkar, Zesheng Xu +6
We introduce a novel robotic system for improving unseen object instance segmentation in the real world by leveraging long-term robot interaction with objects. Previous approaches…
SLGTformer: An Attention-Based Approach to Sign Language Recognition
Neil Song, Yu Xiang
Sign language is the preferred method of communication of deaf or mute people, but similar to any language, it is difficult to learn and represents a significant barrier for those…
Mean Shift Mask Transformer for Unseen Object Instance Segmentation
Yangxiao Lu, Yuqiao Chen, Nicholas Ruozzi +1
Segmenting unseen objects from images is a critical perception skill that a robot needs to acquire. In robot manipulation, it can facilitate a robot to grasp and manipulate unseen…
NeuralGrasps: Learning Implicit Representations for Grasps of Multiple Robotic Hands
Ninad Khargonkar, Neil Song, Zesheng Xu +2
We introduce a neural implicit representation for grasps of objects from multiple robotic hands. Different grasps across multiple robotic hands are encoded into a shared latent spa…
FewSOL: A Dataset for Few-Shot Object Learning in Robotic Environments
Jishnu Jaykumar P, Yu-Wei Chao, Yu Xiang
We introduce the Few-Shot Object Learning (FewSOL) dataset for object recognition with a few images per object. We captured 336 real-world objects with 9 RGB-D images per object fr…
HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers
Yu-Wei Chao, Chris Paxton, Yu Xiang +6
We introduce a new simulation benchmark "HandoverSim" for human-to-robot object handovers. To simulate the giver's motion, we leverage a recent motion capture dataset of hand grasp…
iCaps: Iterative Category-level Object Pose and Shape Estimation
Xinke Deng, Junyi Geng, Timothy Bretl +2
This paper proposes a category-level 6D object pose and shape estimation approach iCaps, which allows tracking 6D poses of unseen objects in a category and estimating their 3D shap…
TALISMAN: Targeted Active Learning for Object Detection with Rare Classes and Slices using Submodular Mutual Information
Suraj Kothawade, Saikat Ghosh, Sumit Shekhar +2
Deep neural networks based object detectors have shown great success in a variety of domains like autonomous vehicles, biomedical imaging, etc. It is known that their success depen…
Hierarchical Policies for Cluttered-Scene Grasping with Latent Plans
Lirui Wang, Xiangyun Meng, Yu Xiang +1
6D grasping in cluttered scenes is a longstanding problem in robotic manipulation. Open-loop manipulation pipelines may fail due to inaccurate state estimation, while most end-to-e…
RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks
Christopher Xie, Arsalan Mousavian, Yu Xiang +1
Segmenting unseen object instances in cluttered environments is an important capability that robots need when functioning in unstructured environments. While previous methods have…
DexYCB: A Benchmark for Capturing Hand Grasping of Objects
Yu-Wei Chao, Wei Yang, Yu Xiang +9
We introduce DexYCB, a new dataset for capturing hand grasping of objects. We first compare DexYCB with a related one through cross-dataset evaluation. We then present a thorough b…
RGB-D Local Implicit Function for Depth Completion of Transparent Objects
Luyang Zhu, Arsalan Mousavian, Yu Xiang +4
Majority of the perception methods in robotics require depth information provided by RGB-D cameras. However, standard 3D sensors fail to capture depth of transparent objects due to…
Learning Composable Behavior Embeddings for Long-horizon Visual Navigation
Xiangyun Meng, Yu Xiang, Dieter Fox
Learning high-level navigation behaviors has important implications: it enables robots to build compact visual memory for repeating demonstrations and to build sparse topological m…
Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Clouds
Lirui Wang, Yu Xiang, Wei Yang +2
6D robotic grasping beyond top-down bin-picking scenarios is a challenging task. Previous solutions based on 6D grasp synthesis with robot motion planning usually operate in an ope…
Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation
Yu Xiang, Christopher Xie, Arsalan Mousavian +1
Segmenting unseen objects in cluttered scenes is an important skill that robots need to acquire in order to perform tasks in new environments. In this work, we propose a new method…
Unseen Object Instance Segmentation for Robotic Environments
Christopher Xie, Yu Xiang, Arsalan Mousavian +1
In order to function in unstructured environments, robots need the ability to recognize unseen objects. We take a step in this direction by tackling the problem of segmenting unsee…
LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation
Keunhong Park, Arsalan Mousavian, Yu Xiang +1
Current 6D object pose estimation methods usually require a 3D model for each object. These methods also require additional training in order to incorporate new objects. As a resul…
Manipulation Trajectory Optimization with Online Grasp Synthesis and Selection
Lirui Wang, Yu Xiang, Dieter Fox
In robot manipulation, planning the motion of a robot manipulator to grasp an object is a fundamental problem. A manipulation planner needs to generate a trajectory of the manipula…
Scaling Local Control to Large-Scale Topological Navigation
Xiangyun Meng, Nathan Ratliff, Yu Xiang +1
Visual topological navigation has been revitalized recently thanks to the advancement of deep learning that substantially improves robot perception. However, the scalability and re…
Self-supervised 6D Object Pose Estimation for Robot Manipulation
Xinke Deng, Yu Xiang, Arsalan Mousavian +3
To teach robots skills, it is crucial to obtain data with supervision. Since annotating real world data is time-consuming and expensive, enabling robots to learn in a self-supervis…
The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation
Christopher Xie, Yu Xiang, Arsalan Mousavian +1
In order to function in unstructured environments, robots need the ability to recognize unseen novel objects. We take a step in this direction by tackling the problem of segmenting…
PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking
Xinke Deng, Arsalan Mousavian, Yu Xiang +3
Tracking 6D poses of objects from videos provides rich information to a robot in performing different tasks such as manipulation and navigation. In this work, we formulate the 6D o…