TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensors

Abstract

Simulators perform an important role in prototyping, debugging, and benchmarking new advances in robotics and learning for control. Although many physics engines exist, some aspects of the real world are harder than others to simulate. One of the aspects that have so far eluded accurate simulation is touch sensing. To address this gap, we present TACTO - a fast, flexible, and open-source simulator for vision-based tactile sensors. This simulator allows to render realistic high-resolution touch readings at hundreds of frames per second, and can be easily configured to simulate different vision-based tactile sensors, including DIGIT and OmniTact. In this paper, we detail the principles that drove the implementation of TACTO and how they are reflected in its architecture. We demonstrate TACTO on a perceptual task, by learning to predict grasp stability using touch from 1 million grasps, and on a marble manipulation control task. Moreover, we provide a proof-of-concept that TACTO can be successfully used for Sim2Real applications. We believe that TACTO is a step towards the widespread adoption of touch sensing in robotic applications, and to enable machine learning practitioners interested in multi-modal learning and control. TACTO is open-source at https://github.com/facebookresearch/tacto.

arXiv commentAccepted to IEEE RAL and ICRA 2022

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Practical Bayesian Optimization of Machine Learning Algorithms
• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• Solving Rubik's Cube with a Robot Hand
• DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
• The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?

Cited by in corpus (13)

• Visuo-Haptic Object Perception for Robots: An Overview
• When Vision Meets Touch: A Contemporary Review for Visuotactile Sensors from the Signal Processing Perspective
• Tacchi: A Pluggable and Low Computational Cost Elastomer Deformation Simulator for Optical Tactile Sensors
• AllSight: A Low-Cost and High-Resolution Round Tactile Sensor with Zero-Shot Learning Capability
• TacSL: A Library for Visuotactile Sensor Simulation and Learning
• Tac-Man: Tactile-Informed Prior-Free Manipulation of Articulated Objects
• A Survey of Embodied Learning for Object-Centric Robotic Manipulation
• DigiTac: A DIGIT-TacTip Hybrid Tactile Sensor for Comparing Low-Cost High-Resolution Robot Touch
• Vision-based Tactile Image Generation via Contact Condition-guided Diffusion Model
• Optimizing BioTac Simulation for Realistic Tactile Perception
• ACROSS: A Deformation-Based Cross-Modal Representation for Robotic Tactile Perception
• SimTac: A Physics-Based Simulator for Vision-Based Tactile Sensing with Biomorphic Structures
• Towards Learning to Play Piano with Dexterous Hands and Touch