55 citations · 232 across the 30 of their papers we have counts for
30 papers
Dex1B: Learning with 1B Demonstrations for Dexterous Manipulation
Jianglong Ye, Keyi Wang, Chengjing Yuan +6
Generating large-scale demonstrations for dexterous hand manipulation remains challenging, and several approaches have been proposed in recent years to address this. Among them, ge…
ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperation
Shiqi Yang, Minghuan Liu, Yuzhe Qin +6
Learning from demonstrations has shown to be an effective approach to robotic manipulation, especially with the recently collected large-scale robot data with teleoperation systems…
Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning
Runyu Ding, Yuzhe Qin, Jiyue Zhu +5
Teleoperation is a crucial tool for collecting human demonstrations, but controlling robots with bimanual dexterous hands remains a challenge. Existing teleoperation systems strugg…
Part-Guided 3D RL for Sim2Real Articulated Object Manipulation
Pengwei Xie, Rui Chen, Siang Chen +6
Manipulating unseen articulated objects through visual feedback is a critical but challenging task for real robots. Existing learning-based solutions mainly focus on visual afforda…
Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning
Entong Su, Chengzhe Jia, Yuzhe Qin +4
Using tactile sensors for manipulation remains one of the most challenging problems in robotics. At the heart of these challenges is generalization: How can we train a tactile-base…
CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulation
Jun Wang, Yuzhe Qin, Kaiming Kuang +4
We introduce CyberDemo, a novel approach to robotic imitation learning that leverages simulated human demonstrations for real-world tasks. By incorporating extensive data augmentat…
DexTouch: Learning to Seek and Manipulate Objects with Tactile Dexterity
Kang-Won Lee, Yuzhe Qin, Xiaolong Wang +1
The sense of touch is an essential ability for skillfully performing a variety of tasks, providing the capacity to search and manipulate objects without relying on visual informati…
Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing
Ying Yuan, Haichuan Che, Yuzhe Qin +6
Executing contact-rich manipulation tasks necessitates the fusion of tactile and visual feedback. However, the distinct nature of these modalities poses significant challenges. In…
GenSim: Generating Robotic Simulation Tasks via Large Language Models
Lirui Wang, Yiyang Ling, Zhecheng Yuan +6
Collecting large amounts of real-world interaction data to train general robotic policies is often prohibitively expensive, thus motivating the use of simulation data. However, exi…
Dynamic Handover: Throw and Catch with Bimanual Hands
Binghao Huang, Yuanpei Chen, Tianyu Wang +4
Humans throw and catch objects all the time. However, such a seemingly common skill introduces a lot of challenges for robots to achieve: The robots need to operate such dynamic ac…
AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
Yuzhe Qin, Wei Yang, Binghao Huang +5
Vision-based teleoperation offers the possibility to endow robots with human-level intelligence to physically interact with the environment, while only requiring low-cost camera se…
DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects
Chen Bao, Helin Xu, Yuzhe Qin +1
To enable general-purpose robots, we will require the robot to operate daily articulated objects as humans do. Current robot manipulation has heavily relied on using a parallel gri…
ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimation
Zehao Zhu, Jiashun Wang, Yuzhe Qin +3
We propose a new dataset and a novel approach to learning hand-object interaction priors for hand and articulated object pose estimation. We first collect a dataset using visual te…
EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Exploration
Linghao Chen, Yuzhe Qin, Xiaowei Zhou +1
Hand-eye calibration is a critical task in robotics, as it directly affects the efficacy of critical operations such as manipulation and grasping. Traditional methods for achieving…
Rotating without Seeing: Towards In-hand Dexterity through Touch
Zhao-Heng Yin, Binghao Huang, Yuzhe Qin +2
Tactile information plays a critical role in human dexterity. It reveals useful contact information that may not be inferred directly from vision. In fact, humans can even perform…
DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation
Yuzhe Qin, Binghao Huang, Zhao-Heng Yin +2
We propose a sim-to-real framework for dexterous manipulation which can generalize to new objects of the same category in the real world. The key of our framework is to train the m…
Abstract-to-Executable Trajectory Translation for One-Shot Task Generalization
Stone Tao, Xiaochen Li, Tongzhou Mu +3
Training long-horizon robotic policies in complex physical environments is essential for many applications, such as robotic manipulation. However, learning a policy that can genera…
A Real2Sim2Real Method for Robust Object Grasping with Neural Surface Reconstruction
Luobin Wang, Runlin Guo, Quan Vuong +3
Recent 3D-based manipulation methods either directly predict the grasp pose using 3D neural networks, or solve the grasp pose using similar objects retrieved from shape databases.…
Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrations
Jianglong Ye, Jiashun Wang, Binghao Huang +2
We propose to learn to generate grasping motion for manipulation with a dexterous hand using implicit functions. With continuous time inputs, the model can generate a continuous an…
From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation
Yuzhe Qin, Hao Su, Xiaolong Wang
We propose to perform imitation learning for dexterous manipulation with multi-finger robot hand from human demonstrations, and transfer the policy to the real robot hand. We intro…
Close the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation
Xiaoshuai Zhang, Rui Chen, Ang Li +11
In this paper, we focus on the simulation of active stereovision depth sensors, which are popular in both academic and industry communities. Inspired by the underlying mechanism of…
Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization
Chieko Sarah Imai, Minghao Zhang, Yuchen Zhang +4
Developing robust vision-guided controllers for quadrupedal robots in complex environments, with various obstacles, dynamical surroundings and uneven terrains, is very challenging.…
DexMV: Imitation Learning for Dexterous Manipulation from Human Videos
Yuzhe Qin, Yueh-Hua Wu, Shaowei Liu +4
While significant progress has been made on understanding hand-object interactions in computer vision, it is still very challenging for robots to perform complex dexterous manipula…
O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning
Kaichun Mo, Yuzhe Qin, Fanbo Xiang +2
Contrary to the vast literature in modeling, perceiving, and understanding agent-object (e.g., human-object, hand-object, robot-object) interaction in computer vision and robotics,…