From pixels to percepts: Highly robust edge perception and contour following using deep learning and an optical biomimetic tactile sensor

Abstract

Deep learning has the potential to have the impact on robot touch that it has had on robot vision. Optical tactile sensors act as a bridge between the subjects by allowing techniques from vision to be applied to touch. In this paper, we apply deep learning to an optical biomimetic tactile sensor, the TacTip, which images an array of papillae (pins) inside its sensing surface analogous to structures within human skin. Our main result is that the application of a deep CNN can give reliable edge perception and thus a robust policy for planning contact points to move around object contours. Robustness is demonstrated over several irregular and compliant objects with both tapping and continuous sliding, using a model trained only by tapping onto a disk. These results relied on using techniques to encourage generalization to tasks beyond which the model was trained. We expect this is a generic problem in practical applications of tactile sensing that deep learning will solve. A video demonstrating the approach can be found at https://www.youtube.com/watch?v=QHrGsG9AHts

Accepted in RAL and ICRA 2019. N. Lepora and J. Lloyd contributed equally to this work

References in corpus (4)

• More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
• Shape-independent Hardness Estimation Using Deep Learning and a GelSight Tactile Sensor
• The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?
• Principal Components of Touch

Cited by in corpus (13)

• When Vision Meets Touch: A Contemporary Review for Visuotactile Sensors from the Signal Processing Perspective
• Tac2Pose: Tactile Object Pose Estimation from the First Touch
• A Vision-Based Tactile Sensing System for Multimodal Contact Information Perception via Neural Network
• Active Multi-Object Exploration and Recognition via Tactile Whiskers
• Tac-Man: Tactile-Informed Prior-Free Manipulation of Articulated Objects
• Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering
• SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulation
• GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger
• DigiTac: A DIGIT-TacTip Hybrid Tactile Sensor for Comparing Low-Cost High-Resolution Robot Touch
• Optimal Deep Learning for Robot Touch
• Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear
• Vision-Guided Active Tactile Perception for Crack Detection and Reconstruction
• Deep Reinforcement Learning for Tactile Robotics: Learning to Type on a Braille Keyboard