Learning Dexterous Manipulation Policies from Experience and Imitation

Abstract

We explore learning-based approaches for feedback control of a dexterous five-finger hand performing non-prehensile manipulation. First, we learn local controllers that are able to perform the task starting at a predefined initial state. These controllers are constructed using trajectory optimization with respect to locally-linear time-varying models learned directly from sensor data. In some cases, we initialize the optimizer with human demonstrations collected via teleoperation in a virtual environment. We demonstrate that such controllers can perform the task robustly, both in simulation and on the physical platform, for a limited range of initial conditions around the trained starting state. We then consider two interpolation methods for generalizing to a wider range of initial conditions: deep learning, and nearest neighbors. We find that nearest neighbors achieve higher performance. Nevertheless, the neural network has its advantages: it uses only tactile and proprioceptive feedback but no visual feedback about the object (i.e. it performs the task blind) and learns a time-invariant policy. In contrast, the nearest neighbors method switches between time-varying local controllers based on the proximity of initial object states sensed via motion capture. While both generalization methods leave room for improvement, our work shows that (i) local trajectory-based controllers for complex non-prehensile manipulation tasks can be constructed from surprisingly small amounts of training data, and (ii) collections of such controllers can be interpolated to form more global controllers. Results are summarized in the supplementary video: https://youtu.be/E0wmO6deqjo

Initial draft for a journal submission

References in corpus (1)

• End-to-End Training of Deep Visuomotor Policies

Cited by in corpus (15)

• Solving Rubik's Cube with a Robot Hand
• Learning Dexterous In-Hand Manipulation
• Reinforcement and Imitation Learning for Diverse Visuomotor Skills
• Active Learning in Robotics: A Review of Control Principles
• Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
• COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning
• ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations
• Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning
• Real-Time Model Calibration with Deep Reinforcement Learning
• Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitation
• Interpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programming
• A Workflow for Offline Model-Free Robotic Reinforcement Learning
• Embodied Language Grounding with 3D Visual Feature Representations
• Battery Model Calibration with Deep Reinforcement Learning
• Autonomous Functional Locomotion in a Tendon-Driven Limb via Limited Experience