Learning Purely Tactile In-Hand Manipulation with a Torque-Controlled Hand

Abstract

We show that a purely tactile dextrous in-hand manipulation task with continuous regrasping, requiring permanent force closure, can be learned from scratch and executed robustly on a torque-controlled humanoid robotic hand. The task is rotating a cube without dropping it, but in contrast to OpenAI's seminal cube manipulation task, the palm faces downwards and no cameras but only the hand's position and torque sensing are used. Although the task seems simple, it combines for the first time all the challenges in execution as well as learning that are important for using in-hand manipulation in real-world applications. We efficiently train in a precisely modeled and identified rigid body simulation with off-policy deep reinforcement learning, significantly sped up by a domain adapted curriculum, leading to a moderate 600 CPU hours of training time. The resulting policy is robustly transferred to the real humanoid DLR Hand-II, e.g., reaching more than 46 full 2$\pi$ rotations of the cube in a single run and allowing for disturbances like different cube sizes, hand orientation, or pulling a finger.

References in corpus (2)

• Surprisingly Robust In-Hand Manipulation: An Empirical Study
• On Policy Learning Robust to Irreversible Events: An Application to Robotic In-Hand Manipulation

Cited by in corpus (8)

• Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes
• Estimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulation
• Combining Shape Completion and Grasp Prediction for Fast and Versatile Grasping with a Multi-Fingered Hand
• Self-Contained and Automatic Calibration of a Multi-Fingered Hand Using Only Pairwise Contact Measurements
• Composing Dextrous Grasping and In-hand Manipulation via Scoring with a Reinforcement Learning Critic
• Neuromorphic force-control in an industrial task: validating energy and latency benefits
• TEXterity -- Tactile Extrinsic deXterity: Simultaneous Tactile Estimation and Control for Extrinsic Dexterity
• Trajectory Optimization for In-Hand Manipulation with Tactile Force Control