Publications (24)
ViTac: Feature Sharing between Vision and Tactile Sensing for Cloth Texture Recognition
Shan Luo, Wenzhen Yuan, Edward Adelson +2
Vision and touch are two of the important sensing modalities for humans and they offer complementary information for sensing the environment. Robots could also benefit from such mu…
Tactile-Reactive Roller Grasper
Shenli Yuan, Shaoxiong Wang, Radhen Patel +4
Manipulation of objects within a robot's hand is one of the most important challenges in achieving robot dexterity. The "Roller Graspers" refers to a family of non-anthropomorphic…
Slip Detection with Combined Tactile and Visual Information
Jianhua Li, Siyuan Dong, Edward Adelson
Slip detection plays a vital role in robotic manipulation and it has long been a challenging problem in the robotic community. In this paper, we propose a new method based on deep…
PolyTouch: A Robust Multi-Modal Tactile Sensor for Contact-rich Manipulation Using Tactile-Diffusion Policies
Jialiang Zhao, Naveen Kuppuswamy, Siyuan Feng +2
Achieving robust dexterous manipulation in unstructured domestic environments remains a significant challenge in robotics. Even with state-of-the-art robot learning methods, haptic…
You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile Contact
Pengfei Ye, Yuxiang Ma, Haonan Chen +5
Accurate 6-DoF object pose estimation is fundamental to robotic manipulation, yet vision-based methods often fail under occlusion, poor lighting, and reflective or transparent surf…
EyeSight Hand: Design of a Fully-Actuated Dexterous Robot Hand with Integrated Vision-Based Tactile Sensors and Compliant Actuation
Branden Romero, Hao-Shu Fang, Pulkit Agrawal +1
In this work, we introduce the EyeSight Hand, a novel 7 degrees of freedom (DoF) humanoid hand featuring integrated vision-based tactile sensors tailored for enhanced whole-hand ma…
Visuotactile Affordances for Cloth Manipulation with Local Control
Neha Sunil, Shaoxiong Wang, Yu She +2
Cloth in the real world is often crumpled, self-occluded, or folded in on itself such that key regions, such as corners, are not directly graspable, making manipulation difficult.…
Connecting Look and Feel: Associating the visual and tactile properties of physical materials
Wenzhen Yuan, Shaoxiong Wang, Siyuan Dong +1
For machines to interact with the physical world, they must understand the physical properties of objects and materials they encounter. We use fabrics as an example of a deformable…
SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulation
Chen Wang, Shaoxiong Wang, Branden Romero +2
Several robot manipulation tasks are extremely sensitive to variations of the physical properties of the manipulated objects. One such task is manipulating objects by using gravity…
DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulation
Hao-Shu Fang, Branden Romero, Yichen Xie +9
We introduce perioperation, a paradigm for robotic data collection that sensorizes and records human manipulation while maximizing the transferability of the data to real robots. W…
Soft, Round, High Resolution Tactile Fingertip Sensors for Dexterous Robotic Manipulation
Branden Romero, Filipe Veiga, Edward Adelson
High resolution tactile sensors are often bulky and have shape profiles that make them awkward for use in manipulation. This becomes important when using such sensors as fingertips…
Exoskeleton-covered soft finger with vision-based proprioception and tactile sensing
Yu She, Sandra Q. Liu, Peiyu Yu +1
Soft robots offer significant advantages in adaptability, safety, and dexterity compared to conventional rigid-body robots. However, it is challenging to equip soft robots with acc…
Design of a Fully Actuated Robotic Hand With Multiple Gelsight Tactile Sensors
Achu Wilson, Shaoxiong Wang, Branden Romero +1
This work details the design of a novel two finger robot gripper with multiple Gelsight based optical-tactile sensors covering the inner surface of the hand. The multiple Gelsight…
See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation
Hao Li, Yizhi Zhang, Junzhe Zhu +7
Humans use all of their senses to accomplish different tasks in everyday activities. In contrast, existing work on robotic manipulation mostly relies on one, or occasionally two mo…
GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger
Elliott Donlon, Siyuan Dong, Melody Liu +3
This work describes the development of a high-resolution tactile-sensing finger for robot grasping. This finger, inspired by previous GelSight sensing techniques, features an integ…
Active Clothing Material Perception using Tactile Sensing and Deep Learning
Wenzhen Yuan, Yuchen Mo, Shaoxiong Wang +1
Humans represent and discriminate the objects in the same category using their properties, and an intelligent robot should be able to do the same. In this paper, we build a robot s…
Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes
Tao Chen, Megha Tippur, Siyang Wu +3
In-hand object reorientation is necessary for performing many dexterous manipulation tasks, such as tool use in less structured environments that remain beyond the reach of current…
GelLink: A Compact Multi-phalanx Finger with Vision-based Tactile Sensing and Proprioception
Yuxiang Ma, Jialiang Zhao, Edward Adelson
Compared to fully-actuated robotic end-effectors, underactuated ones are generally more adaptive, robust, and cost-effective. However, state estimation for underactuated hands is u…
GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger
Shaoxiong Wang, Yu She, Branden Romero +1
Vision-based tactile sensors have the potential to provide important contact geometry to localize the objective with visual occlusion. However, it is challenging to measure high-re…
Cable Manipulation with a Tactile-Reactive Gripper
Yu She, Shaoxiong Wang, Siyuan Dong +3
Cables are complex, high dimensional, and dynamic objects. Standard approaches to manipulate them often rely on conservative strategies that involve long series of very slow and in…
Improved GelSight Tactile Sensor for Measuring Geometry and Slip
Siyuan Dong, Wenzhen Yuan, Edward Adelson
A GelSight sensor uses an elastomeric slab covered with a reflective membrane to measure tactile signals. It measures the 3D geometry and contact force information with high spacia…
Reactive In-Air Clothing Manipulation with Confidence-Aware Dense Correspondence and Visuotactile Affordance
Neha Sunil, Megha Tippur, Arnau Saumell +2
Manipulating clothing is challenging due to complex configurations, variable material dynamics, and frequent self-occlusion. Prior systems often flatten garments or assume visibili…
TactoFind: A Tactile Only System for Object Retrieval
Sameer Pai, Tao Chen, Megha Tippur +3
We study the problem of object retrieval in scenarios where visual sensing is absent, object shapes are unknown beforehand and objects can move freely, like grabbing objects out of…
Digger Finger: GelSight Tactile Sensor for Object Identification Inside Granular Media
Radhen Patel, Rui Ouyang, Branden Romero +1
In this paper we present an early prototype of the Digger Finger that is designed to easily penetrate granular media and is equipped with the GelSight sensor. Identifying objects b…