27 citations · 71 across the 15 of their papers we have counts for
16 papers
9DTact: A Compact Vision-Based Tactile Sensor for Accurate 3D Shape Reconstruction and Generalizable 6D Force Estimation
Changyi Lin, Han Zhang, Jikai Xu +2
The advancements in vision-based tactile sensors have boosted the aptitude of robots to perform contact-rich manipulation, particularly when precise positioning and contact state o…
ArrayBot: Reinforcement Learning for Generalizable Distributed Manipulation through Touch
Zhengrong Xue, Han Zhang, Jingwen Cheng +5
We present ArrayBot, a distributed manipulation system consisting of a array of vertically sliding pillars integrated with tactile sensors, which can simultaneously…
RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools
Haochen Shi, Huazhe Xu, Samuel Clarke +2
Humans excel in complex long-horizon soft body manipulation tasks via flexible tool use: bread baking requires a knife to slice the dough and a rolling pin to flatten it. Often reg…
Scene Synthesis from Human Motion
Sifan Ye, Yixing Wang, Jiaman Li +4
Large-scale capture of human motion with diverse, complex scenes, while immensely useful, is often considered prohibitively costly. Meanwhile, human motion alone contains rich info…
See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation
Hao Li, Yizhi Zhang, Junzhe Zhu +7
Humans use all of their senses to accomplish different tasks in everyday activities. In contrast, existing work on robotic manipulation mostly relies on one, or occasionally two mo…
On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline
Nicklas Hansen, Zhecheng Yuan, Yanjie Ze +5
In this paper, we examine the effectiveness of pre-training for visuo-motor control tasks. We revisit a simple Learning-from-Scratch (LfS) baseline that incorporates data augmentat…
Extraneousness-Aware Imitation Learning
Ray Chen Zheng, Kaizhe Hu, Zhecheng Yuan +2
Visual imitation learning provides an effective framework to learn skills from demonstrations. However, the quality of the provided demonstrations usually significantly affects the…
DTact: A Vision-Based Tactile Sensor that Measures High-Resolution 3D Geometry Directly from Darkness
Changyi Lin, Ziqi Lin, Shaoxiong Wang +1
Vision-based tactile sensors that can measure 3D geometry of the contacting objects are crucial for robots to perform dexterous manipulation tasks. However, the existing sensors ar…
USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulation
Zhengrong Xue, Zhecheng Yuan, Jiashun Wang +3
Can a robot manipulate intra-category unseen objects in arbitrary poses with the help of a mere demonstration of grasping pose on a single object instance? In this paper, we try to…
Phasic Self-Imitative Reduction for Sparse-Reward Goal-Conditioned Reinforcement Learning
Yunfei Li, Tian Gao, Jiaqi Yang +2
It has been a recent trend to leverage the power of supervised learning (SL) towards more effective reinforcement learning (RL) methods. We propose a novel phasic approach by alter…
RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networks
Haochen Shi, Huazhe Xu, Zhiao Huang +2
Modeling and manipulating elasto-plastic objects are essential capabilities for robots to perform complex industrial and household interaction tasks (e.g., stuffing dumplings, roll…
Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learning
Zhecheng Yuan, Guozheng Ma, Yao Mu +5
One of the key challenges in visual Reinforcement Learning (RL) is to learn policies that can generalize to unseen environments. Recently, data augmentation techniques aiming at en…
Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
Ruihan Yang, Minghao Zhang, Nicklas Hansen +2
We propose to address quadrupedal locomotion tasks using Reinforcement Learning (RL) with a Transformer-based model that learns to combine proprioceptive information and high-dimen…
DAIR: Disentangled Attention Intrinsic Regularization for Safe and Efficient Bimanual Manipulation
Minghao Zhang, Pingcheng Jian, Yi Wu +2
We address the problem of safely solving complex bimanual robot manipulation tasks with sparse rewards. Such challenging tasks can be decomposed into sub-tasks that are accomplisha…
Towards Learning to Play Piano with Dexterous Hands and Touch
Huazhe Xu, Yuping Luo, Shaoxiong Wang +2
The virtuoso plays the piano with passion, poetry and extraordinary technical ability. As Liszt said (a virtuoso)must call up scent and blossom, and breathe the breath of life. The…
PyTouch: A Machine Learning Library for Touch Processing
Mike Lambeta, Huazhe Xu, Jingwei Xu +4
With the increased availability of rich tactile sensors, there is an equally proportional need for open-source and integrated software capable of efficiently and effectively proces…