Deep Reinforcement Learning for Tactile Robotics: Learning to Type on a Braille Keyboard

Abstract

Artificial touch would seem well-suited for Reinforcement Learning (RL), since both paradigms rely on interaction with an environment. Here we propose a new environment and set of tasks to encourage development of tactile reinforcement learning: learning to type on a braille keyboard. Four tasks are proposed, progressing in difficulty from arrow to alphabet keys and from discrete to continuous actions. A simulated counterpart is also constructed by sampling tactile data from the physical environment. Using state-of-the-art deep RL algorithms, we show that all of these tasks can be successfully learnt in simulation, and 3 out of 4 tasks can be learned on the real robot. A lack of sample efficiency currently makes the continuous alphabet task impractical on the robot. To the best of our knowledge, this work presents the first demonstration of successfully training deep RL agents in the real world using observations that exclusively consist of tactile images. To aid future research utilising this environment, the code for this project has been released along with designs of the braille keycaps for 3D printing and a guide for recreating the experiments. A brief video summary is also available at https://youtu.be/eNylCA2uE_E.

Accepted in RAL and IROS 2020

References in corpus (15)

• Playing Atari with Deep Reinforcement Learning
• Trust Region Policy Optimization
• Soft Actor-Critic Algorithms and Applications
• Dueling Network Architectures for Deep Reinforcement Learning
• Solving Rubik's Cube with a Robot Hand
• DeepMind Control Suite
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
• Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research
• From pixels to percepts: Highly robust edge perception and contour following using deep learning and an optical biomimetic tactile sensor
• Data-efficient Deep Reinforcement Learning for Dexterous Manipulation
• Benchmarking Reinforcement Learning Algorithms on Real-World Robots
• Learning Gentle Object Manipulation with Curiosity-Driven Deep Reinforcement Learning
• OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research
• Surface Following using Deep Reinforcement Learning and a GelSightTactile Sensor