Event-Driven Visual-Tactile Sensing and Learning for Robots
arXiv:2009.07083
Abstract
This work contributes an event-driven visual-tactile perception system, comprising a novel biologically-inspired tactile sensor and multi-modal spike-based learning. Our neuromorphic fingertip tactile sensor, NeuTouch, scales well with the number of taxels thanks to its event-based nature. Likewise, our Visual-Tactile Spiking Neural Network (VT-SNN) enables fast perception when coupled with event sensors. We evaluate our visual-tactile system (using the NeuTouch and Prophesee event camera) on two robot tasks: container classification and rotational slip detection. On both tasks, we observe good accuracies relative to standard deep learning methods. We have made our visual-tactile datasets freely-available to encourage research on multi-modal event-driven robot perception, which we believe is a promising approach towards intelligent power-efficient robot systems.
RSS 2020, Code and Datasets are available at https://clear-nus.github.io/visuotactile/index.html
References in corpus (12)
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Deep Learning in Spiking Neural Networks
- Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols
- EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
- SLAYER: Spike Layer Error Reassignment in Time
- More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
- Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Study
- Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets
- Deep Learning without Weight Transport
- ViTac: Feature Sharing between Vision and Tactile Sensing for Cloth Texture Recognition
- Slip Detection with Combined Tactile and Visual Information
- Learning to Identify Object Instances by Touch: Tactile Recognition via Multimodal Matching